Skip to main content

Visualization of Graph Databases Using Cytoscape.js

(UPDATED APR. 2024)  I have ample evidence from multiple sources that there are strong unmet needs in the area of visualization of graph databases.

And whenever there's a vacuum, vendors circle like vultures - with incomplete, non-customizable, and at times ridiculously expensive, closed-box proprietary solutions.
 
Fortunately, coming to the rescue is the awesome open-source cytoscape.js library,  an offshoot of the "Cytoscape" project of the Institute for Systems Biology, a project with a long history that goes back to 2002.

One can do amazing custom solutions, relatively easily, when one combines this Cytoscape library with:
 
1) a front-end framework such as Vue.js
 
2) backend libraries (for example in python) to prepare and serve the data
 
For example, a while back I created a visualizer for networks of chemical reactions, for another open-source project I lead (life123.science)
 
This visualizer will look and feel generally familiar to anyone who has ever used the standard Neo4j web browser, but it's totally under your control :  you can expand it as it suits you (for example, to add editing capabilities), and you can have it within your own web app.
 
Click her for --->  LIVE DEMO  
 
A screenshot: 

This article is a detailed "under-the-hood" view.  Future articles will provide simplified summaries.

This article is part of a growing, ongoing series on Graph Databases and Neo4j

A Minimalist Example

Let's temporarily backtrack to an even simpler example, to drive home how ridiculously easy it is to use Cytoscape.js 

Here's the creation of a simple interactive visualization, with just a few lines of code in an HTML file (the heavy lifting being done behind-the-scene by the Cytoscape.js library):


And here's the self-contained HTML file, same as shown in the above picture, with just a couple of extra lines to make it valid HTML.  Just download it to your desktop!  And then, yes, you can just double-click on it (or drag it to a browser) - and open it in your browser, with the interactive graph.

Division of Responsibilities

To keep in mind: cytoscape.js is just a "scaffold" to do the basic heavy lifting; you can make it to be as fancy as you want by personalizing it.

You can bring in:
 
1) the power of the renowned open-source visualization library D3.js, to do a fine levels of styling 
 
2) your favorite front-end framework (such as Vue.js or React) to facilitate other interactive support, such as menus and all sort of page elements for display and editing)
 
3) your favorite back-end framework (such as Flask or FastAPI) to query and prepare the data to pass to the front-end, as well as possibly manage database changes requested by the front end
 
With a relatively brief involvement by a full-stack developer, possibly even a junior one (or two separate developers for the front- and back- ends) you can implement just about everything you need and can imagine - finely "custom fitted", just about any way you need it!

Going Beyond Minimalist

In the previous section, we discussed a minimalist example of a tiny HTML file (which you can download to your desktop, and open with a browser) that uses only a few lines of JavaScript, to drive the embedded Cytoscape.js library to create a minimalist - but fully interactive - graph. 
 
The burning question is how to progress beyond that? 
 
The answer, to me, is to have:
 
1) a convenient backend python library to prepare the data for the frontend module. 
Typically, data scientists wanting to precise manage what to display will deal with just this part.
 
2) a deeply customizable frontend module, in the framework of your choice, to wrap around Cytoscape, and provide whatever display and controls you want on your UI - exquisitely customized to your needs like a tailor-fitted suit!  
The heavy lifting (the interactive "balls and links") is handled by Cytoscape; so, your front-end can concentrate on just whatever other bells and whistles you need on that page - such as forms and menus.
Typically, this module will be created by a front-end programmer : one module for each different use case that requires high customization.  Generic versions of this module (such as  the ones about to be released by the BrainAnnex open-source project) will suffice for a lot other cases.

What's Available?

A backend python library, and a front-end module (using the Vue.js framework, which in principle could be ported to another framework)... became available with BrainAnnex' recent  Beta 37 release, which is probably the last of the beta releases - since the project is getting much more stable, and ready to leave beta stage!

Here's an example of a screenshot from an early test of a Schema Viewer (an optional web app that is distributed alongside the foundational libraries of the Brain Annex project), with extra debugging information that demonstrates the handover of data (top box) from the backend to the front end:
You can use your own favorite technology stack, but this particular UI page is generated using Flask.

With Flask, inside the HTML file template (to be precise a "Jinja" template) there's a handover of the data from the backend to the front end : a Python dictionary gets converted to a JavaScript object, for use by the Vue.js object on that page.

Then, the JavaScript data gets passed to a module (a "Vue.js component") that embeds Cytoscape.js.  
Inside that module, Cytoscape manages the interactive "balls and arrows" (large shaded area at the bottom, in above screenshot) while the customizable rest of the Vue module manages the legend box (where it's saying "id: 1, labels: PERSON", etc) and whatever other feature we want - for example to edit graph data.  User edits result in update requests that the frontend sends to the server thru a web API.

Putting it All Together

(APRIL 2023 UPDATE) With the latest release of the open-source "BrainAnnex" libraries, one has full control, with considerable ease, of both the backend and the front-end, for :
  • Data preparation
  • Data update
  • Serving the data, and offering updating operations, with a web API
  • A powerful standard front-end framework to orchestrate all sorts of UI elements  (Vue.js, but in principle you could port that to your favorite framework)
  • Cytoscape.js to manage the low-level interactive ball-and-rods displays
Voila', armed with that apparatus, it was fairly easy to implement a feature inside a web app : to display the "Schema" currently stored in the graph database ("Schema" is the data-integrity, etc, layer described here; but you can just think of it as a subset of the database.)
 
The current UI design opted to provide the ability to switch between 2 initial views at the click of a button (red highlight).   
 
View 1 ("breadth first"):

View 2 ("random"):

The default initial views of the graph are entirely managed by the Cytoscape.js library (I rearranged a handful of nodes by hand prior to taking the screenshots.)   More such views are available, and custom ones can be created; currently exploring these possibilities!

The button to switch the initial view is managed by the front-end framework (red box).  Likewise, when a node is clicked on (yellow highlight in View1, near the top), Cytoscape makes the data available, and the front-end framework creates the legend box at the top/right.

The legend box could be modified to provide editing capabilities as well.  Future releases of Brain Annex will explore that...
In short:  (Total Control of the Backend and Frontend) + (Visualization Details Managed from Cytoscape) = ONE CAN IMPLEMENT RELATIVELY EASILY ALL SORTS OF CUSTOM VISUALIZATION  
You can look under the hood in the Flask template file for this Schema Viewer.   That file is largely the HMTL scaffolding; the actual work - and the interface with the Cytoscape library - takes places inside the Vue component (module.)

Is Your Head Spinning?

This is an under-the-hood look at both the back- and front-end...  perhaps covering too much!  Future articles will digest individual parts.
Also, if you "just want something to use out of the box", a future article will address that!

Resources

Python library "PyGraphScape" : repository

Vue component "vue_cytoscape_2" repository

Cytoscape.js library official page

A great overview of Cytoscape

Under-the-hood Cytoscape:


This article is part of a growing, ongoing series on Graph Databases and Neo4j

Comments

Popular posts from this blog

Discussing Neuroscience with ChatGPT

UPDATED Apr. 2023 - I'm excited by ChatGPT 's possibilities in terms of facilitating advanced learning .  For example, I got enlightening answers to questions that I had confronted when I first studied neuroscience.  The examples below are taken from a very recent session I had with ChatGPT (mid Jan. 2023.) Source: https://neurosciencestuff.tumblr.com In case you're not familiar with ChatGPT, it's a very sophisticated "chatbot" - though, if you call it that way, it'll correct you!  'I am not a "chatbot", I am a language model, a sophisticated type of AI algorithm trained on vast amounts of text data to generate human-like text'. For a high-level explanation of how ChatGPT actually works - which also gives immense insight into its weaknesses, there's an excellent late Jan. 2023 talk by Stephen Wolfram, the brilliant author of the Mathematica software and of Wolfram Alpha , a product that could be combined with ChatGPT to imp

Using Neo4j with Python : the Open-Source Library "NeoAccess"

So, you want to build a python app or Jupyter notebook to utilize Neo4j, but aren't too keen on coding a lot of string manipulation to programmatic create ad-hoc Cypher queries?   You're in the right place: the NeoAccess library can do take care of all that, sparing you from lengthy, error-prone development that requires substantial graph-database and software-development expertise! This article is part 4 of a growing,  ongoing  series  on Graph Databases and Neo4j   "NeoAccess" is the bottom layer of the technology stack provided by the BrainAnnex open-source project .  All layers are very modular, and the NeoAccess library may also be used by itself , entirely separately from the rest of the technology stack.  (A diagram of the full stack is shown later in this article.) NeoAccess interacts with the Neo4j Python driver , which is provided by the Neo4j company, to access the database from Python; the API to access that driver is very powerful, but complex - and does

Graph Databases (Neo4j) - a revolution in modeling the real world!

UPDATED Oct. 2023 - I was "married" to Relational Databases for many years... and it was a good "relationship" full of love and productivity - but SOMETHING WAS MISSING! Let me backtrack.   In college, I got a hint of the "pre-relational database" days...  Mercifully, that was largely before my time, but  - primarily through a class - I got a taste of what the world was like before relational databases.  It's an understatement to say: YUCK! Gratitude for the power and convenience of Relational Databases and SQL - and relief at having narrowly averted life before it! - made me an instant mega-fan of that technology.  And for many years I held various jobs that, directly or indirectly, made use of MySQL and other relational databases - whether as a Database Administrator, Full-Stack Developer, Data Scientist, CTO or various other roles. UPDATE: This article is now part 1 of a growing, ongoing series on Graph Databases and Neo4j But ther

What are Graph Databases - and Why Should I Care?? : "Graph Databases for Poets"

  This is a very gentle introduction to the subject.  The subtitle is inspired by university courses such as "Physics for Poets"!  (if you're technically inclined, there's an alternate article for you.) It has been said that "The language of physics (or of God) is math".  On a similar note, it could be said that: The language of the biological world - or of any subject or endeavor involving complexity - is networks ('meshes') What is a network?  Think of  it as the familiar 'friends of friends' diagram from social media. Everywhere one turns in biology, there's a network – at the cellular level, tissue level, organ level, ecosystem level.  The weather and other earth systems are networks.  Human societal organization is a network.  Electrical circuits, the Internet, our own brains...  Networks are everywhere! What can we do with networks, to better understand the world around us, or to create something that we need? Broadly s

Full-Text Search with the Neo4j Graph Database

(UPDATED Oct. 2023)   Now that we have discussed a full technology stack based on Neo4j (or other graph databases), and that we a design and implementation available from the open-source project BrainAnnex.org  , what next?  What shall we build on top? Well, how about  Full-Text Search ?  This article is part of a growing, ongoing series on Graph Databases and Neo4j Full-Text Searching/Indexing Starting with the  Version 5, Beta 26.1  release, the Brain Annex open-source project includes a straightforward but working implementation of a design that uses the convenient services of its Schema Layer , to provide indexing of word-based documents using Neo4j. The python class FullTextIndexing ( source code ) provides the necessary methods, and it can parse both plain-text and HTML documents (for example, used in "formatted notes"); parsing of PDF files and other formats will be added at a later date. No grammatical analysis ( stemming or lemmatizing ) is done on

Using Schema in Graph Databases such as Neo4j

UPDATED Feb. 2024 - Graph databases have an easygoing laissez-faire attitude: "express yourself (almost) however you want"... By contrast, relational databases come across with an attitude like a micro-manager:  "my way or the highway"... Is there a way to take the best of both worlds and distance oneself from their respective excesses, as best suited for one's needs?  A way to marry the flexibility of Graph Databases and the discipline of Relational Databases? This article is part 5 of a growing,  ongoing  series  on Graph Databases and Neo4j Let's Get Concrete Consider a simple scenario with scientific data such as the Sample, Experiment, Study, Run Result , where Samples are used in Experiments, and where Experiments are part of Studies and produce Run Results.  That’s all very easy and intuitive to represent and store in a Labeled Graph Database such as Neo4j .   For example, a rough draft might go like this:   The “labels” (black tags) represent

Anti-Aging Research: Science, not Hype

Last updated May 2023 Q: "How is aging a disease?" A: It's a dynamic system that veers away from its homeostasis (normal equilibrium point): hence a form of slow-progressing illness. Labeling it as 'natural' is a surrender to our traditional state of ignorance and powerlessness, which fortunately is beginning to be changed! Aging is "normal" only from the point of view of the "selfish gene", for whom the body is a disposable carrier. Individuals organisms - for whom self-preservation has a different meaning than for genes - have received scant help from evolution... with rare exceptions such as the T. dohrnii jellyfish (which I discuss here )... but now the time has finally arrived for our rational design to remedy some of the cellular flaws that evolution never bothered to correct!   The above is my standard answer to an oft-asked question. The science of aging is by all evidence very misunderstood by the general public.  Hype,

Neo4j & Cypher Tutorial : Getting Started with a Graph Database and its Query Language

You have a general idea of what Graph Databases - and Neo4j in particular - are...  But how to get started?  Read on! This article is part 3 of a growing,  ongoing  series  on Graph Databases and Neo4j   If you're new to graph databases, please check out part 1 for an intro and motivation about them.  There, we discussed an example about an extremely simple database involving actors, movies and directors...  and saw how easy the Cypher query language makes it to answer questions such as "which directors have worked with Tom Hanks in 2016" - questions that, when done with relational databases and SQL, turn into a monster of a query and an overly-complicated data model involving a whopping 5 tables! In this tutorial, we will actually carry out that query - and get acquainted with Cypher and the Neo4j browser interface in the process.  This is the dataset we'll be constructing: Get the database in place If you don't already have a database installed locally

Neo4j Sandbox Tutorial : try Neo4j and learn Cypher - free and easy!

So, you have an itch to test-drive Neo4j and its Cypher query language.  Maybe you want to learn it, or evaluate it, or introduce colleagues/clients to it.  And you wish for: fast, simple and free! Well, good news: the Neo4j company kindly provides a free, short-term hosted solution called "the Neo4j sandbox" .  Extremely easy to set up and use! This article is part 2 of a growing, ongoing series on Graph Databases and Neo4j Register (free) for the Neo4j "Sandbox" Go to sandbox.neo4j.com , and register with a working email and a password.  That's it! Note that this same email/password will also let you into the Neo4j Community Forums and Support ; the same login for all: very convenient! Launch your instance - blank or pre-populated After registering, go to  sandbox.neo4j.com  , and follow the steps in the diagram below (the choices might differ, but the "Blank Sandbox" should always be there): Too good to be true?  Is there