Whole brain microscopy analysis

With BrainGlobe and napari

Welcome

Schedule (day 1)

• Installing BrainGlobe and Napari (30 mins)

• Introduction to Image analysis with Napari (60 mins)

• Introduction to BrainGlobe (30 mins)

• Registering whole brain microscopy images with brainreg (30 mins)

• Segmenting structures in whole brain microscopy images with brainglobe-segmentation (30 mins)

• Detecting cells in large 3D images with cellfinder (90 mins)

• Combining brainreg and cellfinder on the command line (brainmapper)(30 mins)

• Unstructured time (debugging problems, networking, discussions) (60 mins)

Schedule (day 2)

• Analysing brainmapper outputs in napari (30 mins)

• Visualising atlases with brainrender-napari (30 mins)

• Visualising cells in atlas space (30 mins)

• Visualising cell density with brainglobe-heatmap (30 mins)

• Scripting a visualisation with brainrender (30 mins)

• Tour of other BrainGlobe tools (30 mins)

• Where to get help (30 mins)

• Contributing to BrainGlobe and the Scientific Python ecosystem (30 mins)

• Contibuting an atlas to BrainGlobe (30 mins)

• Unstructured time (debugging problems, networking, discussions, scripting) (60 mins)

• Hackday ideas (30 mins)

Installation

Install napari and BrainGlobe

Everyone

conda create -n brainglobe-env python=3.12
conda activate brainglobe-env
pip install brainglobe

Silicon mac users (additionally)

conda install niftyreg

Double-check installation

Double-check that running

napari

opens a new napari window, with brainglobe plugins available under Plugins.

Install BrainGlobe atlases

Run

brainglobe install -a allen_mouse_50um
brainglobe install -a allen_mouse_10um
brainglobe install -a ccfv3augmented_mouse_25um

To check whether this worked:

brainglobe list

Image Analysis (with napari)

Adapted from https://github.com/HealthBioscienceIDEAS/microscopy-novice/ (under CC BY 4.0 license)

Why Napari

• Napari is a graphical user interface
• All BrainGlobe analyses have a napari plugin

Opening Napari

napari

Opening images

File > Open Files(s), then navigate to calcium imaging folder and open translation1_00001_ce.tif

If napari asks, select napari builtins as the reader in the pop-up window

Napari’s User interface

Canvas

Try moving around the image with the following commands:

Pan - Click and drag
Zoom - Scroll in/out

Dimension sliders

Dimension sliders appear at the bottom of the canvas depending on the type of image displayed. For example, here we have a 3D image of some cells, which consists of a stack of 2D images. If we drag the slider at the bottom of the image, we move up and down in this stack:

Pressing the arrow buttons at either end of the slider steps through one slice at a time. Also, pressing the ‘play’ button at the very left of the slider moves automatically through the stack until pressed again.

We will see in later episodes that more sliders can appear if our image has more dimensions (e.g. time series, or further channels).

Viewer buttons

The viewer buttons (the row of buttons at the bottom left of Napari) control various aspects of the Napari viewer:

• Console

• 2D/3D /

• Roll dimensions

• Transpose dimensions

• Grid

• Home

Layer list

We can show/hide each layer by clicking the eye icon on the left side of their row. We can also rename them by double clicking on the row.

We can change the order of layers by dragging and dropping items in the layer list. For example, try dragging the membrane layer above the nuclei. You should see the nuclei disappear from the viewer (as they are now hidden by the membrane image on top).

Now that we’ve seen the main controls for the viewer, let’s look at the layer list. ‘Layers’ are how Napari displays multiple items together in the viewer. For example, currently our layer list contains two items - ‘nuclei’ and ‘membrane’. These are both Image layers and are displayed in order, with the nuclei on top and membrane underneath.

Here we only have Image layers, but there are many more types like Points, Shapes and Labels, some of which we will see later in the episode.

Layer controls

This area shows controls only for the currently selected layer (i.e. the one that is highlighted in blue in the layer list).

• Opacity
  
• Contrast limits
• …

This changes the opacity of the layer - lower values are more transparent. For example, reducing the opacity of the membrane layer (if it is still on top of the nuclei), allows us to see the nuclei again.

briefly - the contrast limits adjust what parts of the image we can see and how bright they appear in the viewer. Moving the left node adjusts what is shown as fully black, while moving the right node adjusts what is shown as fully bright. This doesn’t change the values in the image, just how they are displayed.

Layer buttons

Create and remove.

• Points

• Shapes

• Labels

• Remove layer
  

Other layer types

Note that there are some layer types that can’t be added via clicking buttons in the user interface, like

• surfaces,
• tracks and
• vectors.

These require calling python commands in Napari’s console or an external python script.

Key points

• Napari’s user interface is split into a few main sections including the canvas, layer list, layer controls…
• Layers can be of different types e.g. Image, Point, Label
• Different layer types have different layer controls

A deeper look

File > Open Files(s), then navigate to calcium imaging folder and open translation1_00001_ce.tif

Pixels

This 2D image shows the nuclei of human cells undergoing mitosis. If we really zoom in up-close by scrolling, we can see that this image is actually made up of many small squares with different brightness values. These squares are the image’s pixels (or ‘picture elements’) and are the individual units that make up all digital images.

If we hover over these pixels with the mouse cursor, we can see that each pixel has a specific value. Try hovering over pixels in dark and bright areas of the image and see how the value changes in the bottom left of the viewer:

Images are arrays of numbers

Open Napari’s built-in console and run:

# Get the image data for the first layer in Napari
image = viewer.layers[0].data

# Print the image values and type
print(image)
print(type(image))

We’ve seen that images are made of individual units called pixels that have specific values - but how is an image really represented in the computer? Let’s dig deeper into Napari’s Image layers…

All of the information about the Napari viewer can be accessed through the console with a variable called viewer. A viewer has 1 to many layers, and here we access the top (first) layer with viewer.layers[0]. Then, to access the actual image data stored in that layer, we retrieve it with .data:

You should see that a series of numbers are printed out that are stored in a Python data type called a numpy.ndarray. Fundamentally, this means that all images are really just arrays of numbers (one number per pixel). Arrays are just rectangular grids of numbers, much like a spreadsheet. Napari is reading those values and converting them into squares of particular colours for us to see in the viewer, but this is only to help us interpret the image contents - the numbers are the real underlying data.

In Napari this array is a numpy.ndarray. NumPy is a popular python package that provides ‘n-dimensional arrays’ (or ‘ndarray’ for short). N-dimensional just means they can support any number of dimensions - for example, 2D (squares/rectangles of numbers), 3D (cubes/cuboids of numbers) and beyond (like time series, images with many channels etc. where we would have multiple rectangles or cuboids of data which provide further information all at the same location).

Image dimensions

We can find out the dimensions by running the following in Napari’s console:

image.shape

Let’s return to our image and explore some of the key features of its image array. First, what size is it?

The array size (also known as its dimensions) is stored in the .shape. Three values are printed as this image is three dimensional (3D), for a 2D image there would be 2, for a 4D image there would be 4 and so on…

Image data type

The other key feature of an image array is its ‘data type’ - this controls which values can be stored inside of it.

image.dtype.name

The data type consists of type and bit-depth.

For example, let’s look at the data type for our image - this is stored in .dtype:

We see that the data type (or ‘dtype’ for short) is uint16. This is short for ‘unsigned integer 16-bit’. Let’s break this down further into two parts - the type (unsigned integer) and the bit-depth (16-bit).

Type

The type determines what kind of values can be stored in the array, for example:

• Unsigned integer: positive whole numbers
• Signed integer: positive and negative whole numbers
• Float: positive and negative numbers with a decimal point e.g. 3.14

For our calcium image, ‘unsigned integer’ means that only positive whole numbers can be stored inside. You can see this by hovering over the pixels in the image again in Napari - the pixel value down in the bottom left is always a positive whole number.

Bit depth

The bit depth determines the range of values that can be stored e.g. only values between 0 and \(2^{16}-1\).

print(image.min())
print(image.max())

Common data types

NumPy supports a very wide range of data types, but there are a few that are most common for image data:

NumPy datatype	Full name	Range of values
uint8	Unsigned integer 8-bit	0…255
uint16	Unsigned integer 16-bit	0…65535
float32	Float 32-bit	\(-3.4 \times 10^{38}...+3.4 \times 10^{38}\)
float64	Float 64-bit	\(-1.7 \times 10^{308}...+1.7 \times 10^{308}\)

uint8 and uint16 are most common for images from light microscopes. float32 and float64 are common during image processing (as we will see in later episodes).

Coordinate system

• For 2d images, y is the first coordinate
• For 3d images, z is the first coordinate

We’ve seen that images are arrays of numbers with a specific shape (dimensions) and data type. How do we access specific values from this array? What coordinate system is Napari using?

As you move around, you should see that the lowest coordinate values are at the top left corner, with the first value increasing as you move down and the second value increasing as you move to the right. This is different to the standard coordinate systems you may be used to (for example, from making graphs):

Note that Napari lists coordinates as [y, x] or [rows, columns], so e.g. [1,3] would be the pixel in row 1 and column 3. Remember that these coordinates always start from 0 as you can see in the diagram:

Key points

• Digital images are made of pixels
• Digital images store these pixels as arrays of numbers
• Napari (and Python more widely) use NumPy arrays to store images - these have a shape and dtype
• Most images are 8-bit or 16-bit unsigned integer
• Images use a coordinate system with (0,0) at the top left, x increasing to the right, and y increasing down

Processing images

Now we understand what an image is, and how to look at it in napari, we can start measuring things! But we need to find (“segment”) “things” first!

Reduce noise with a median filter

from scipy.signal import medfilt2d
image = viewer.layers[0].data
filtered = medfilt2d(image)
viewer.add_image(filtered)

Isolate neurons with a threshold

Example of “semantic” segmentation

thresholded = filtered > 8000 # True or False array
viewer.add_image(thresholded)

Label each neuron with a number

Example of “instance” segmentation

from skimage.measure import regionprops, label
labelled = label(thresholded)
viewer.add_labels(labelled)

Pixels in each neuron

properties = regionprops(labelled)
pixels_in_each_region = [prop.area for prop in properties]
print(pixels_in_each_region)

Key points

• Segmentation can be broadly split into ‘semantic segmentation’ (e.g. neuron vs background) and ‘instance segmentation’ (e.g. individual neuron).
• Segmentations are represented in the computer in the same way as images, but pixels represent an abstraction, rather than light intensity.
• Napari uses “Labels” layers for segmentations.
• Segmentation is helpful for analysis.

BrainGlobe

BrainGlobe Initiative

Established 2020 with three aims:

1. Develop general-purpose tools to help others build interoperable software
2. Develop specialist software for specific analysis and visualisation needs
3. Reduce barriers of entry, and facilitate the building of an ecosystem of computational neuroanatomy tools.

BrainGlobe Atlas API

Initial observation - lots of similar communities working independently

• Model species
• Imaging modality
• Anatomical focus
• Developmental stage

BrainGlobe Atlas API

brainglobe-atlasapi

Current atlases

Atlas Name	Resolution	Ages	Reference Images
Allen Mouse Brain Atlas	10, 25, 50, and 100 micron	P56	STPT
Allen Human Brain Atlas	100 micron	Adult	MRI
Max Planck Zebrafish Brain Atlas	1 micron	6-dpf	FISH
Enhanced and Unified Mouse Brain Atlas	10, 25, 50, and 100 micron	P56	STPT
Smoothed version of the Kim et al. mouse reference atlas	10, 25, 50 and 100 micron	P56	STPT
Gubra’s LSFM mouse brain atlas	20 micron	8 to 10 weeks post natal	LSFM
3D version of the Allen mouse spinal cord atlas	20 x 10 x 10 micron	Adult	Nissl
AZBA: A 3D Adult Zebrafish Brain Atlas	4 micron	15-16 weeks post natal	LSFM
Waxholm Space atlas of the Sprague Dawley rat brain	39 micron	P80	MRI
3D Edge-Aware Refined Atlases Derived from the Allen Developing Mouse Brain Atlases	16, 16.75, and 25 micron	E13, E15, E18, P4, P14, P28 & P56	Nissl
Princeton Mouse Brain Atlas	20 micron	>P56 (older animals included)	LSFM
Kim Lab Developmental CCF	10 micron	P56	STP, LSFM (iDISCO) and MRI (a0, adc, dwo, fa, MTR, T2)
Blind Mexican Cavefish Brain Atlas	2 micron	1 year	IHC
BlueBrain Barrel Cortex Atlas	10 and 25 micron	P56	STPT
UNAM Axolotl Brain Atlas	40 micron	~ 3 months post hatching	MRI

BrainGlobe Atlas API

from brainglobe_atlasapi import BrainGlobeAtlas
atlas = BrainGlobeAtlas("allen_mouse_25um")

reference_image = atlas.reference
print(reference_image.shape)
# (528, 320, 456)

annotation_image = atlas.annotation
print(annotation_image.shape)
# (528, 320, 456)

from pprint import pprint
VISp = atlas.structures["VISp"]
pprint(VISp)

# {'acronym': 'VISp',
#  'id': 385,
#  'mesh': None,
#  'mesh_filename': PosixPath('/home/user/.brainglobe/allen_mouse_25um_v0.3/meshes/385.obj'),
#  'name': 'Primary visual area',
#  'rgb_triplet': [8, 133, 140],
#  'structure_id_path': [997, 8, 567, 688, 695, 315, 669, 385]}

Version 1

Whole brain microscopy

Serial section two-photon tomography

Fluorescence micro-optical sectioning tomography

Light sheet fluorescence microscopy

Whole-brain registration

brainreg

3D cell detection

cellfinder

3D cell detection

cellfinder

3D cell detection

cellfinder

3D cell detection

cellfinder

Spatial analysis

brainglobe-segmentation

Spatial analysis

brainglobe-segmentation

Spatial analysis

brainglobe-segmentation

Visualisation

brainrender

Visualisation

brainrender

Version 2

Expanding access

Adding new atlases

• Latest version of Waxholm rat
• Axolotl
• Prarie Vole
• Princeton RAtlas
• Cuttlefish
• Developmental Mouse Brain atlas
• NHP
• Human

Building novel atlases

More raw data processing

Consistent napari environment

Support for more data types

BrainGlobe website

Tutorials

Motivation

By the end of the course (maybe on your own data):

Motivation

Tutorials (day 1)

• registering whole-brain microscopy to an atlas
• cell detection in whole-brain microscopy
  • retraining cellfinder to fine-tune cell detection
• combining registration and cell detection

Data

• feel free to experiment with your own data
• small sample data comes with napari
• large sample data MS_cx_left
  • on HD: share

MS_cx_left data courtesy of Brown APY, Cossell L,Strom M, Tyson AL, Vélez-Fort M, Margrie TW. Analysis of segmentationontology reveals the similarities and differences in connectivity onto L2/3 neurons in mouse V1. Scientific Reports. 2021; 11 (4983). https://doi.org/10.1038/s41598-021-82353-7

Quick excursion into BrainGlobe orientation

• three letters define position of pixel (0,0,0)
• e.g. psl - pixel (0,0,0) is Posterior, Superior, Left
• all BrainGlobe atlases are asr
• to verify: open data in napari and scroll through data
  • demo

Registering whole brain microscopy images with brainreg

Tutorial

Segmenting structures in whole brain microscopy images with brainglobe-segmentation

Tutorial (1D)

Tutorial (2D/3D)

Detecting cells in large 3D images with cellfinder

Cell detection tutorial

Retraining tutorial

Pulling things together: brainmapper CLI

• brainmapper is the name of our command line tool to combine cell detection and atlas registration.

brainmapper CLI

• Required arguments
  • -s The primary signal channel
  • -b The secondary autofluorescence channel
  • -o The output directory
  • --orientation e.g. `psl
  • -v The voxel spacing (microns) in the same order as the data orientation (psl): 5 2 2.
  • --atlas

Quick reminder of BrainGlobe orientation

• three letters define position of pixel (0,0,0)
• e.g. psl - pixel (0,0,0) is Posterior, Superior, Left
• all BrainGlobe atlases are asr
• to verify: open data in napari and scroll through data

Day 2

Schedule (day 1)

• Installing BrainGlobe and Napari (30 mins)

• Introduction to Image analysis with Napari (60 mins)

• Introduction to BrainGlobe (30 mins)

• Registering whole brain microscopy images with brainreg (30 mins)

• Segmenting structures in whole brain microscopy images with brainglobe-segmentation (30 mins)

• Detecting cells in large 3D images with cellfinder (90 mins)

• Combining brainreg and cellfinder on the command line (brainmapper)(30 mins)

• Unstructured time (debugging problems, networking, discussions) (60 mins)

Schedule (day 2)

• Analysing brainmapper outputs in napari (30 mins)

• Visualising atlases with brainrender-napari (30 mins)

• Visualising cells in atlas space (30 mins)

• Visualising cell density with brainglobe-heatmap (30 mins)

• Scripting a visualisation with brainrender (30 mins)

• Tour of other BrainGlobe tools (30 mins)

• Where to get help (30 mins)

• Contributing to BrainGlobe and the Scientific Python ecosystem (30 mins)

• Contibuting an atlas to BrainGlobe (30 mins)

• Unstructured time (debugging problems, networking, discussions, scripting) (60 mins)

• Hackday ideas (30 mins)

Tutorials (day 2)

Sample data

• Use the folder open-software-week/brainglobe-course-data/MS_CX_left_cellfinder
• It is the output of brainmapper on MS_cx_left data

MS_cx_left data courtesy of Brown APY, Cossell L,Strom M, Tyson AL, Vélez-Fort M, Margrie TW. Analysis of segmentationontology reveals the similarities and differences in connectivity onto L2/3 neurons in mouse V1. Scientific Reports. 2021; 11 (4983). https://doi.org/10.1038/s41598-021-82353-7

Analysing cell positions in atlas space

Tutorial

Visualisation of data in atlas space with brainrender-napari

Tutorial (brainrender-napari)

Making heatmaps with brainglobe-heatmap scripts

Task: * Visualise the cell density/region of a coronal slice

Example brainglobe-heatmap scripts

Making renders with brainrender scripts

Tasks: * Visualise a region in the atlas * Visualise the cells from the large sample data * (Bonus) Make an animation

Example brainrender scripts

Tour of other BrainGlobe tools and wrap-up

• brainglobe-atlasapi
• brainglobe-registration
• brainglobe-space
• brainglobe-stitch
• brainglobe-template-builder
• brainglobe-utils
• morphapi

How to get help

• BrainGlobe website
• Help forum
• Developer forum
• GitHub

You are welcome to contribute to BrainGlobe - get in touch anytime and we will support you!

BrainGlobe and the scientific Python ecosystem

BrainGlobe and the scientific Python ecosystem

BrainGlobe makes the scientific Python ecosystem accessible to neuroscientists.

BrainGlobe and the scientific Python ecosystem

We depend on the scientific Python ecosystem.

The open-source ecology

E.g. brainglobe-atlasapi

A selection of tools built by the community

Other neuroscience tools depend on BrainGlobe

You are part of the ecosystem

• we welcome contributions
  • raise issues
  • write tutorials
  • developer guide
  • we can mentor

Some wider starting points

• guide to first open-source contributions
• good first issues in open-source

Contributing atlases to BrainGlobe

Before starting, you will need:

• Information to credit the atlas (publication, website)
• A template reference 3D stack (at least one)
• An annotation 3D stack (at finest level of your region hierarchy)
• A description of all the structures of the brain and their hierarchy.

How do I contribute an atlas

• Write a packaging script
• Submit a Pull Request
• Core developers upload to data platform

How are atlases stored?

• On “GIN” (German Informatics Node)
  • acts as an archive of atlas data
• improvements (Atlas API V2) in progress
  • use zarr to “stream” atlases
  • also avoid data duplication

Contributing atlases to BrainGlobe

More info on the BrainGlobe website. Achievable (?) hackday idea:

• Pick an atlas of interest
• Intermediate Python knowledge
• template script available
• Bonus: improve related documentation

Thank you!

Please fill out the feedback form.