NWB Widget Demo

NWB Widgets¶

NWB Widgets is a python package for automatic, interactive, performant exploration of data in NWB files. This notebook demonstrates how to use NWB Widgets to explore data, and how to stream data from the DANDI archive directly into NWB Widgets.

NWB Widgets uses the metadata of the NWB file to understand the contents and infer what visualizations make sense for this data. The code is exactly the same for visualizing each different NWB filea. We demonstrate this here with one calcium imaging NWB file and one with Neuropixel extracellular electrophysiology.

While this notebook can be run on a properly configured environment anywhere, it will be particularly easy to use and performant using the "NWBstream" environment deployed for free on DANDI Hub.

In [ ]:

import pynwb
from pynwb import NWBHDF5IO
from nwbwidgets import nwb2widget
import requests

def _search_assets(url, filepath):
    response = requests.request("GET", url, headers={"Accept": "application/json"}).json() 
    
    for asset in response["results"]:
        if filepath == asset["path"]:
            return asset["asset_id"]
    
    if response.get("next", None):
        return _search_assets(response["next"], filepath)
    
    raise ValueError(f'path {filepath} not found in dandiset {dandiset_id}.')


def get_asset_id(dandiset_id, filepath):
    url = f"https://api.dandiarchive.org/api/dandisets/{dandiset_id}/versions/draft/assets/"
    return _search_assets(url, filepath)

def get_s3_url(dandiset_id, filepath):
    """Get the s3 location for any NWB file on DANDI"""

    asset_id = get_asset_id(dandiset_id, filepath)
    url = f"https://api.dandiarchive.org/api/dandisets/{dandiset_id}/versions/draft/assets/{asset_id}/download/"
    
    s3_url = requests.request(url=url, method='head').url
    if '?' in s3_url:
        return s3_url[:s3_url.index('?')]
    return s3_url

import pynwb from pynwb import NWBHDF5IO from nwbwidgets import nwb2widget import requests def _search_assets(url, filepath): response = requests.request("GET", url, headers={"Accept": "application/json"}).json() for asset in response["results"]: if filepath == asset["path"]: return asset["asset_id"] if response.get("next", None): return _search_assets(response["next"], filepath) raise ValueError(f'path {filepath} not found in dandiset {dandiset_id}.') def get_asset_id(dandiset_id, filepath): url = f"https://api.dandiarchive.org/api/dandisets/{dandiset_id}/versions/draft/assets/" return _search_assets(url, filepath) def get_s3_url(dandiset_id, filepath): """Get the s3 location for any NWB file on DANDI""" asset_id = get_asset_id(dandiset_id, filepath) url = f"https://api.dandiarchive.org/api/dandisets/{dandiset_id}/versions/draft/assets/{asset_id}/download/" s3_url = requests.request(url=url, method='head').url if '?' in s3_url: return s3_url[:s3_url.index('?')] return s3_url

In [ ]:

# calcium imaging, Giocomo Lab (30 GB)
dandiset_id, filepath = "000054", "sub-F2/sub-F2_ses-20190407T210000_behavior+ophys.nwb"

# neuropixel, Giocomo Lab (46 GB)
#dandiset_id, filepath = "000053", "sub-npI1/sub-npI1_ses-20190415_behavior+ecephys.nwb"

s3_path = get_s3_url(dandiset_id, filepath)

# calcium imaging, Giocomo Lab (30 GB) dandiset_id, filepath = "000054", "sub-F2/sub-F2_ses-20190407T210000_behavior+ophys.nwb" # neuropixel, Giocomo Lab (46 GB) #dandiset_id, filepath = "000053", "sub-npI1/sub-npI1_ses-20190415_behavior+ecephys.nwb" s3_path = get_s3_url(dandiset_id, filepath)

Note that these NWB files are quite large. In fact, we have chosen NWB files that contain raw data to demonstrate how data streaming can efficiently deal with these large files. Streaming works efficiently with NWB Widgets so that only the data necessary to create each view is read from DANDI. As a result, data transfer is minimized and data can be explored efficiently.

In [ ]:

# use the "Read Only S3" (ros3) driver to stream data directly from DANDI (or any other S3 location)
io = NWBHDF5IO(s3_path, mode='r', load_namespaces=True, driver='ros3')

nwb = io.read()
nwb2widget(nwb)

# use the "Read Only S3" (ros3) driver to stream data directly from DANDI (or any other S3 location) io = NWBHDF5IO(s3_path, mode='r', load_namespaces=True, driver='ros3') nwb = io.read() nwb2widget(nwb)

Running NWBWidgets locally¶

You can also download NWB files and run NWB Widgets locally by passing the local path (and omitting the ros3 driver flag). Here we will demonstrate this with a smaller intracellular electrophysiology dataset.

In [ ]:

# icephys, optogenetics, and behavior, Svoboda Lab (632 MB)
dandiset_id, filepath = "000005", "sub-anm324650/sub-anm324650_ses-20160422_behavior+icephys+ogen.nwb"
asset_id = get_asset_id(dandiset_id, filepath)
print(f"https://api.dandiarchive.org/api/dandisets/{dandiset_id}/versions/draft/assets/{asset_id}/download/")

# icephys, optogenetics, and behavior, Svoboda Lab (632 MB) dandiset_id, filepath = "000005", "sub-anm324650/sub-anm324650_ses-20160422_behavior+icephys+ogen.nwb" asset_id = get_asset_id(dandiset_id, filepath) print(f"https://api.dandiarchive.org/api/dandisets/{dandiset_id}/versions/draft/assets/{asset_id}/download/")

click the link that is output above ^^

In [ ]:

import os.path as op
io = NWBHDF5IO(op.expanduser("~/Downloads/sub-anm324650_ses-20160422_behavior+icephys+ogen.nwb"), mode='r', load_namespaces=True)

nwb = io.read()
nwb2widget(nwb)

import os.path as op io = NWBHDF5IO(op.expanduser("~/Downloads/sub-anm324650_ses-20160422_behavior+icephys+ogen.nwb"), mode='r', load_namespaces=True) nwb = io.read() nwb2widget(nwb)

When running with local data, NWBWidgets still only reads the data that is necessary from disk.