A Technology Stack on Top of a (Neo4j) Graph Database

Putting it All Together : a Technology Stack on top of a (Neo4j) Graph Database

The above technology stack is explained in this short video.

For many practical use cases, one needs a full data-management solution, not just a database.

So, armed with the Schema Layer discussed in the previous part, the next natural step is to add an API and possibly a UI.

This is part 6 of a 7-part series on Graph Databases and Neo4j.

part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world!

part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher the free and easy way

part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language

part 4 : Using Neo4j with Python : the Open-Source Library NeoAccess

part 5 : Using Schema in Graph Databases such as Neo4j

part 6 : Putting it All Together - a Technology Stack on Top of a (Neo4j) Graph Database

part 7 : (SPECIAL TOPIC) Full-Text Search with the Neo4j Graph Database

The API Layer / Data Manager

The API Layer ("Data Manager") is in some ways the most straightforward layer - because the "heavy lifting" is done by the Schema Layer.

I've been involved in projects that utilized an inadequate Schema Layer - and in those situations the API Layer ends up taking on an immense amount of responsibility that don't logically belong there; the end result being a lot of difficult, error-prone and non-modular development that feels like "pulling teeth"!

By contrast, with a well-developed Schema Layer that provides a wide range of services, it becomes a relative breeze to build an API Layer. The API Layer can then concentrate on roles that logically belong there, such as receiving JSON commands, parsing them, and invoking the appropriate methods of the Schema Layer, which does most of the work.

With such a well-defined modular role, the development focus of API Layer can be to provide a set of useful, convenient, and intuitive request-handling commands, typically for the UI and/or to provide a web endpoint.

The BrainAnnex.org project uses the Flask platform to provide a number of typical operations in this layer - such as to add or edit nodes in the database - and it's fairly easy to add additional operations... or duplicate this layer to meet one's needs.

The API Layer can also take on responsibilities such as:

authentication
implementing a master-slave architecture that doesn't require direct Neo4j database support (which would necessitate the licensed version of Neo4j.)

For more information about this layer, please see this short video.

The User Interface Layer

The API Layer takes care of the data management; so that the UI can focus on the front-end user experience.

As far as I'm concerned, excellence in the UI is truly "the final frontier"!

That's the layer that ultimately makes the fundamental difference for the user. Most users won't care if the system has "the engine of a Ferrari", if the vehicle looks and feels like a beaten-up junk car!

All the other layers are in place, and - while there's plenty of room for improvement (and for more documentation) - the foundations are fairly solid and extensive. The UI is a different story; it's the cutting edge of development - and a lot of possibilities exist to explore.

The BrainAnnex.org project uses the Vue.js platform and the Cytoscape.js library to provide a dual modality for displaying/editing data: tabular form and graph form ("balls and edges".)

Also, BrainAnnex makes use of a Class-based UI, whereby some of the data Classes (as defined in the Schema layer) are associated with ad-hoc software ("plugins") to display/edit data of that class in a particular manner. Historically, the early versions of BrainAnnex utilized this approach to implement a multimedia content management system - but now that's just one use case.

The UI Layer is a very active area of development. Just like for the API Layer, one might opt to supplement, based on one's needs, the existing layer provided by BrainAnnex - or replace it with a custom one.

Things start getting especially exciting and powerful when the UI is aware of the different data types (for example, as specified in the Schema layer described in the previous section), and has the capability to personalize the display and editing mode of data records based on their types ("classes") - perhaps with plugins to provide modularity and easy expansion.

Well, that's exactly what the new version (5) of the open-source project Brain Annex does! As of Aug. 2023, it's in a late Beta stage - with the exception of the bottom-most layer, the NeoAccess library (which exited Beta.)

One use case of such a system is to be a multimedia content management system, which is what the old ("vintage") versions of Brain Annex were, before switching to being Neo4j-based, which leads to a far more general system.

Armed with an API, and possibly a UI, one can for example create a standalone web app, or a control panel for an existing website or web app. For example, the diagram at the very top of this page is how Brain Annex does it.

For more information about this layer, please see this short video.

Special-Purpose Modules

Now that we have a whole technology stack, how about looking into special-purpose modules? We'll do that in part 7, with a discussion (and available open-source implementation) of full-text search!

This is part 6 of a 7-part series on Graph Databases and Neo4j.

part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world!

part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher the free and easy way

part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language

part 4 : Using Neo4j with Python : the Open-Source Library NeoAccess

part 5 : Using Schema in Graph Databases such as Neo4j

part 6 : Putting it All Together - a Technology Stack on Top of a (Neo4j) Graph Database

part 7 : (SPECIAL TOPIC) Full-Text Search with the Neo4j Graph Database

Comments

Discussing Neuroscience with ChatGPT

UPDATED Apr. 2023 - I'm excited by ChatGPT 's possibilities in terms of facilitating advanced learning . For example, I got enlightening answers to questions that I had confronted when I first studied neuroscience. The examples below are taken from a very recent session I had with ChatGPT (mid Jan. 2023.) Source: https://neurosciencestuff.tumblr.com In case you're not familiar with ChatGPT, it's a very sophisticated "chatbot" - though, if you call it that way, it'll correct you! 'I am not a "chatbot", I am a language model, a sophisticated type of AI algorithm trained on vast amounts of text data to generate human-like text'. For a high-level explanation of how ChatGPT actually works - which also gives immense insight into its weaknesses, there's an excellent late Jan. 2023 talk by Stephen Wolfram, the brilliant author of the Mathematica software and of Wolfram Alpha , a product that could be combined with ChatGPT to imp

Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language

You have a general idea of what Graph Databases - and Neo4j in particular - are... But how to get started? Read on! This is part 3 of a 7-part series on Graph Databases and Neo4j. part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world! part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher the free and easy way part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language part 4 : Using Neo4j with Python : the Open-Source Library NeoAccess part 5 : Using Schema in Graph Databases such as Neo4j part 6 : Putting it All Together - a Technology Stack on Top of a Graph Database part 7 : (SPECIAL TOPIC) Full-Text Search with the Neo4j Graph Database If you're new to graph databases, please check out part 1 for an intro and motivation about them. There, we discussed an example about an extremely simple database involving actors, movies and directors... and saw how easy the Cypher query lan

Graph Databases (Neo4j) - a revolution in modeling the real world!

UPDATED Oct. 2023 - I was "married" to Relational Databases for many years... and it was a good "relationship" full of love and productivity - but SOMETHING WAS MISSING! Let me backtrack. In college, I got a hint of the "pre-relational database" days... Mercifully, that was largely before my time, but - primarily through a class - I got a taste of what the world was like before relational databases. It's an understatement to say: YUCK! Gratitude for the power and convenience of Relational Databases and SQL - and relief at having narrowly averted life before it! - made me an instant mega-fan of that technology. And for many years I held various jobs that, directly or indirectly, made use of MySQL and other relational databases - whether as a Database Administrator, Full-Stack Developer, Data Scientist, CTO or various other roles. UPDATE: This article is now part of a series... This is part 1 of an ongoing series on Graph Databases and Neo4j.

Using Neo4j with Python : the Open-Source Library "NeoAccess"

So, you want to build a python app or Jupyter notebook to utilize Neo4j, but aren't too keen on coding a lot of string manipulation to programmatic create ad-hoc Cypher queries? You're in the right place: the NeoAccess library can do take care of all that, sparing you from lengthy, error-prone development that requires substantial graph-database and software-development expertise! This is part 4 of a 7-part series on Graph Databases and Neo4j. part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world! part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher – free and easy part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language part 4 : Using Neo4j with Python : the Open-Source Library "NeoAccess" part 5 : Using Schema in Graph Databases such as Neo4j part 6 : Putting it All Together - a Technology Stack on Top of a Graph Database part 7 : (SPECIAL TOPIC) Ful

Full-Text Search with the Neo4j Graph Database

(UPDATED Oct. 2023) In part 5 ( Using Schema in Graph Databases ) we discussed the concept of a Schema Layer, and a design and implementation available from the open-source project BrainAnnex.org Now that we have such a layer, what shall be build on top of it? Well, how about Full-Text Search ? This is part 7 of an ongoing series on Graph Databases and Neo4j. part 0 (alternate version of part 1 for a general, non-technical audience) : What are Graph Databases - and Why Should I Care?? : "Graph Databases for Poets" part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world! part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher the free and easy way part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language part 4 : Using Neo4j with Python : the Open-Source Library NeoAccess part 5 : Using Schema in Graph Databases such as Neo4j part 6 : Putting it All Together - a Technology Stack on Top of a

Using Schema in Graph Databases such as Neo4j

UPDATED Aug. 2023 - Graph databases have an easygoing laissez-faire attitude: "express yourself (almost) however you want"... By contrast, relational databases come across with an attitude along the lines of a micro-manager: "my way or the highway"... Is there a way to take the best of both worlds and distance oneself from their respective excesses, as best suited for one's needs? This is part 5 of a 7-part series on Graph Databases and Neo4j. part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world! part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher the free and easy way part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language part 4 : Using Neo4j with Python : the Open-Source Library NeoAccess part 5 : Using Schema in Graph Databases such as Neo4j part 6 : Putting it All Together - a Technology Stack on Top of a Graph Database part 7 : (SPECIAL TOPIC) F

What are Graph Databases - and Why Should I Care?? : "Graph Databases for Poets"

This is a very gentle introduction to the subject. The subtitle is inspired by university courses such as "Physics for Poets"! (if you're technically inclined, there's an alternate article for you.) It has been said that "The language of physics (or of God) is math". On a similar note, it could be said that: The language of the biological world - or of any subject or endeavor involving complexity - is networks ('meshes') What is a network? Think of it as the familiar 'friends of friends' diagram from social media. Everywhere one turns in biology, there's a network – at the cellular level, tissue level, organ level, ecosystem level. The weather and other earth systems are networks. Human societal organization is a network. Electrical circuits, the Internet, our own brains... Networks are everywhere! What can we do with networks, to better understand the world around us, or to create something that we need? Broadly s

To Build or Not to Build One’s Own Desktop Computer?

“ VALENTINA ” [UPDATED JUNE 2021] - Whether you're a hobbyist, or someone who just needs a good desktop computer, or an IT professional who wants a wider breath of knowledge, or a gamer who needs a performant machine, you might have contemplated at some point whether to build your own desktop computer. If you're a hobbyist, I think it's a great project. If you're an IT professional - especially a "coder" - I urge you to do it: in my opinion, a full-fledged Computer Scientist absolutely needs breath, ranging from the likes of Shannon's Information Theory and the Halting Problem - all the way down to how transistors work. And what about someone who just needs a good desktop computer? A big maybe on that - but perhaps this blog entry will either help you, or scare you off for your own good! To build, or not to build, that is the question: Whether 'tis nobler in the mind to suffer The slings and arrows of OEM's cutting corners and limit

Neo4j Sandbox Tutorial : try Neo4j and learn Cypher - free and easy!

So, you have an itch to test-drive Neo4j and its Cypher query language. Maybe you want to learn it, or evaluate it, or introduce colleagues/clients to it. And you wish for: fast, simple and free! Well, good news: the Neo4j company kindly provides a free, short-term hosted solution called "the Neo4j sandbox" . Extremely easy to set up and use! This is part 2 of a 7-part series on Graph Databases and Neo4j. part 1 : Graph Databases (Neo4j) - a revolution in modeling the real world! part 2 : Neo4j Sandbox Tutorial : try Neo4j and learn Cypher the free and easy way part 3 : Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language part 4 : Using Neo4j with Python : the Open-Source Library NeoAccess part 5 : Using Schema in Graph Databases such as Neo4j part 6 : Putting it All Together - a Technology Stack on Top of a Graph Database part 7 : (SPECIAL TOPIC) Full-Text Search with the Neo4j Graph Database Reg

RDF Triple Stores vs. Property Graphs : How to Attach Properties to Relationships

Time for the opening shot of a series about Semantic Technology , and in particular contrasting-and-comparing the opposing (but perhaps ultimately complementary) camps of: RDF Triple Stores , aka Triples-Based Graphs. For example, Blazegraph or Apache Jena (Labeled) Property Graphs . For example, Neo4j or Blazegraph (For this article, I'll assume that you have at least a passing acquaintance with both. Here is background info on Triplestores and Property Graphs ) It’s my opinion that modeling in terms of Subject/Predicate/Object triples (aka RDF ) might be appealing to mathematicians or philosophers for its minimalist foundation (though a lot of baroque add-on’s quickly come out of the closet!) Modeling in terms of (Labeled) Property Graphs might be appealing to computer scientists, because such graphs appear more usable and less clunky once you start actually doing something with them. Perhaps because I straddle both the Math and CS camps, I’m currently on t

Julian's Polymath Explorations

Search This Blog