{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Channel Quality Assessment and Pruning\n", "\n", "This notebook sketches how to prune bad channels " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2026-05-07T06:38:27.310196Z", "iopub.status.busy": "2026-05-07T06:38:27.307770Z", "iopub.status.idle": "2026-05-07T06:38:27.322858Z", "shell.execute_reply": "2026-05-07T06:38:27.321917Z" } }, "outputs": [], "source": [ "# This cells setups the environment when executed in Google Colab.\n", "try:\n", " import google.colab\n", " !curl -s https://raw.githubusercontent.com/ibs-lab/cedalion/dev/scripts/colab_setup.py -o colab_setup.py\n", " # Select branch with --branch \"branch name\" (default is \"dev\")\n", " %run colab_setup.py\n", "except ImportError:\n", " pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Background: Quality Masks and Channel Pruning\n", "\n", "Cedalion represents data quality as **boolean masks**: `xr.DataArray` objects with `True` meaning *clean* and `False` meaning *tainted*. Multiple masks can be computed from different metrics (SCI, SNR, peak spectral power ratio, channel distance) and stored together in `rec.masks`.\n", "\n", "The key function for applying masks is `cedalion.sigproc.quality.prune_ch()`, which takes a timeseries and a dictionary of masks and either:\n", "- **drops** channels that are flagged bad (removes them from the array), or\n", "- **NaN-fills** those channels (preserves array shape but marks values as missing).\n", "\n", "The `operator` argument controls how multiple masks are combined:\n", "- `\"all\"` — a channel is removed only if *all* masks flag it bad\n", "- `\"any\"` — a channel is removed if *any* mask flags it bad (more aggressive)\n", "\n", "The `dim_collapse` argument (e.g. `\"channel\"`) specifies which dimension to reduce to, collapsing `True`/`False` across wavelengths before applying the prune." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2026-05-07T06:38:27.325148Z", "iopub.status.busy": "2026-05-07T06:38:27.324449Z", "iopub.status.idle": "2026-05-07T06:38:30.399770Z", "shell.execute_reply": "2026-05-07T06:38:30.399270Z" } }, "outputs": [], "source": [ "import matplotlib.pyplot as p\n", "import numpy as np\n", "from matplotlib.colors import LinearSegmentedColormap\n", "\n", "import cedalion\n", "import cedalion.data as datasets\n", "import cedalion.nirs\n", "from cedalion.vis.anatomy import scalp_plot\n", "from cedalion.vis.colors import threshold_cmap, mask_cmap\n", "import cedalion.sigproc.quality as quality\n", "import cedalion.xrutils as xrutils\n", "from cedalion import units\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading raw CW-NIRS data from a SNIRF file and converting it to OD and CONC\n", "\n", "This notebook uses a finger-tapping dataset in BIDS layout provided by Rob Luke that is automatically fetched. \n", "You can also find it [here](https://github.com/rob-luke/BIDS-NIRS-Tapping)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2026-05-07T06:38:30.405410Z", "iopub.status.busy": "2026-05-07T06:38:30.402279Z", "iopub.status.idle": "2026-05-07T06:38:31.456837Z", "shell.execute_reply": "2026-05-07T06:38:31.454772Z" } }, "outputs": [ { "data": { "text/html": [ "
<xarray.DataArray (channel: 28, wavelength: 2, time: 23239)> Size: 10MB\n",
"<Quantity([[[0.0913686 0.0909875 0.0910225 ... 0.0941083 0.0940129 0.0944882]\n",
" [0.1856806 0.186377 0.1836514 ... 0.1856486 0.1850836 0.1842172]]\n",
"\n",
" [[0.227516 0.2297024 0.2261366 ... 0.2264519 0.2271665 0.226713 ]\n",
" [0.6354927 0.637668 0.6298023 ... 0.6072068 0.6087293 0.6091066]]\n",
"\n",
" [[0.1064704 0.1066212 0.1053444 ... 0.121114 0.1205022 0.1205441]\n",
" [0.2755033 0.2761615 0.2727006 ... 0.2911952 0.2900544 0.2909847]]\n",
"\n",
" ...\n",
"\n",
" [[0.2027881 0.1996586 0.2004866 ... 0.2318743 0.2311941 0.2330808]\n",
" [0.4666358 0.4554404 0.4561614 ... 0.4809749 0.4812827 0.4862896]]\n",
"\n",
" [[0.4885007 0.4802285 0.4818338 ... 0.6109142 0.6108118 0.613845 ]\n",
" [0.8457658 0.825988 0.8259648 ... 0.975894 0.9756599 0.9826459]]\n",
"\n",
" [[0.6304559 0.6284427 0.6287045 ... 0.6810626 0.6809573 0.6818709]\n",
" [1.2285622 1.2205907 1.2190002 ... 1.2729124 1.2727222 1.2755645]]], 'volt')>\n",
"Coordinates:\n",
" * time (time) float64 186kB 0.0 0.128 0.256 ... 2.974e+03 2.974e+03\n",
" samples (time) int64 186kB 0 1 2 3 4 5 ... 23234 23235 23236 23237 23238\n",
" * channel (channel) object 224B 'S1D1' 'S1D2' 'S1D3' ... 'S8D8' 'S8D16'\n",
" source (channel) object 224B 'S1' 'S1' 'S1' 'S1' ... 'S8' 'S8' 'S8'\n",
" detector (channel) object 224B 'D1' 'D2' 'D3' 'D9' ... 'D7' 'D8' 'D16'\n",
" * wavelength (wavelength) float64 16B 760.0 850.0\n",
"Attributes:\n",
" data_type_group: unprocessed raw<xarray.DataArray (channel: 24, wavelength: 2, time: 23239)> Size: 9MB\n",
"<Quantity([[[0.0913686 0.0909875 0.0910225 ... 0.0941083 0.0940129 0.0944882]\n",
" [0.1856806 0.186377 0.1836514 ... 0.1856486 0.1850836 0.1842172]]\n",
"\n",
" [[0.227516 0.2297024 0.2261366 ... 0.2264519 0.2271665 0.226713 ]\n",
" [0.6354927 0.637668 0.6298023 ... 0.6072068 0.6087293 0.6091066]]\n",
"\n",
" [[0.1064704 0.1066212 0.1053444 ... 0.121114 0.1205022 0.1205441]\n",
" [0.2755033 0.2761615 0.2727006 ... 0.2911952 0.2900544 0.2909847]]\n",
"\n",
" ...\n",
"\n",
" [[0.187484 0.1868235 0.1866562 ... 0.1735965 0.1736705 0.1738339]\n",
" [0.2424386 0.241503 0.2408491 ... 0.22303 0.2229887 0.2234081]]\n",
"\n",
" [[0.2027881 0.1996586 0.2004866 ... 0.2318743 0.2311941 0.2330808]\n",
" [0.4666358 0.4554404 0.4561614 ... 0.4809749 0.4812827 0.4862896]]\n",
"\n",
" [[0.6304559 0.6284427 0.6287045 ... 0.6810626 0.6809573 0.6818709]\n",
" [1.2285622 1.2205907 1.2190002 ... 1.2729124 1.2727222 1.2755645]]], 'volt')>\n",
"Coordinates:\n",
" * time (time) float64 186kB 0.0 0.128 0.256 ... 2.974e+03 2.974e+03\n",
" samples (time) int64 186kB 0 1 2 3 4 5 ... 23234 23235 23236 23237 23238\n",
" * channel (channel) object 192B 'S1D1' 'S1D2' 'S1D3' ... 'S8D7' 'S8D16'\n",
" source (channel) object 192B 'S1' 'S1' 'S1' 'S1' ... 'S7' 'S8' 'S8'\n",
" detector (channel) object 192B 'D1' 'D2' 'D3' 'D9' ... 'D15' 'D7' 'D16'\n",
" * wavelength (wavelength) float64 16B 760.0 850.0\n",
"Attributes:\n",
" data_type_group: unprocessed raw<xarray.DataArray (channel: 20, wavelength: 2, time: 23239)> Size: 7MB\n",
"<Quantity([[[0.0913686 0.0909875 0.0910225 ... 0.0941083 0.0940129 0.0944882]\n",
" [0.1856806 0.186377 0.1836514 ... 0.1856486 0.1850836 0.1842172]]\n",
"\n",
" [[0.227516 0.2297024 0.2261366 ... 0.2264519 0.2271665 0.226713 ]\n",
" [0.6354927 0.637668 0.6298023 ... 0.6072068 0.6087293 0.6091066]]\n",
"\n",
" [[0.1064704 0.1066212 0.1053444 ... 0.121114 0.1205022 0.1205441]\n",
" [0.2755033 0.2761615 0.2727006 ... 0.2911952 0.2900544 0.2909847]]\n",
"\n",
" ...\n",
"\n",
" [[0.2225884 0.2187791 0.2195495 ... 0.2564863 0.2551258 0.2560233]\n",
" [0.3994258 0.3917637 0.389261 ... 0.4304597 0.430814 0.4331249]]\n",
"\n",
" [[0.2027881 0.1996586 0.2004866 ... 0.2318743 0.2311941 0.2330808]\n",
" [0.4666358 0.4554404 0.4561614 ... 0.4809749 0.4812827 0.4862896]]\n",
"\n",
" [[0.4885007 0.4802285 0.4818338 ... 0.6109142 0.6108118 0.613845 ]\n",
" [0.8457658 0.825988 0.8259648 ... 0.975894 0.9756599 0.9826459]]], 'volt')>\n",
"Coordinates:\n",
" * time (time) float64 186kB 0.0 0.128 0.256 ... 2.974e+03 2.974e+03\n",
" samples (time) int64 186kB 0 1 2 3 4 5 ... 23234 23235 23236 23237 23238\n",
" * channel (channel) object 160B 'S1D1' 'S1D2' 'S1D3' ... 'S8D7' 'S8D8'\n",
" source (channel) object 160B 'S1' 'S1' 'S1' 'S2' ... 'S7' 'S8' 'S8'\n",
" detector (channel) object 160B 'D1' 'D2' 'D3' 'D1' ... 'D7' 'D7' 'D8'\n",
" * wavelength (wavelength) float64 16B 760.0 850.0\n",
"Attributes:\n",
" data_type_group: unprocessed raw<xarray.DataArray (channel: 23, wavelength: 2, time: 23239)> Size: 9MB\n",
"<Quantity([[[0.227516 0.2297024 0.2261366 ... 0.2264519 0.2271665 0.226713 ]\n",
" [0.6354927 0.637668 0.6298023 ... 0.6072068 0.6087293 0.6091066]]\n",
"\n",
" [[0.1064704 0.1066212 0.1053444 ... 0.121114 0.1205022 0.1205441]\n",
" [0.2755033 0.2761615 0.2727006 ... 0.2911952 0.2900544 0.2909847]]\n",
"\n",
" [[0.5512474 0.5510672 0.5476283 ... 0.6179242 0.6188702 0.6187721]\n",
" [1.125532 1.1238331 1.1119423 ... 1.1817728 1.1819598 1.1832658]]\n",
"\n",
" ...\n",
"\n",
" [[0.2027881 0.1996586 0.2004866 ... 0.2318743 0.2311941 0.2330808]\n",
" [0.4666358 0.4554404 0.4561614 ... 0.4809749 0.4812827 0.4862896]]\n",
"\n",
" [[0.4885007 0.4802285 0.4818338 ... 0.6109142 0.6108118 0.613845 ]\n",
" [0.8457658 0.825988 0.8259648 ... 0.975894 0.9756599 0.9826459]]\n",
"\n",
" [[0.6304559 0.6284427 0.6287045 ... 0.6810626 0.6809573 0.6818709]\n",
" [1.2285622 1.2205907 1.2190002 ... 1.2729124 1.2727222 1.2755645]]], 'volt')>\n",
"Coordinates:\n",
" * time (time) float64 186kB 0.0 0.128 0.256 ... 2.974e+03 2.974e+03\n",
" samples (time) int64 186kB 0 1 2 3 4 5 ... 23234 23235 23236 23237 23238\n",
" * channel (channel) object 184B 'S1D2' 'S1D3' 'S2D1' ... 'S8D8' 'S8D16'\n",
" source (channel) object 184B 'S1' 'S1' 'S2' 'S2' ... 'S8' 'S8' 'S8'\n",
" detector (channel) object 184B 'D2' 'D3' 'D1' 'D3' ... 'D7' 'D8' 'D16'\n",
" * wavelength (wavelength) float64 16B 760.0 850.0\n",
"Attributes:\n",
" data_type_group: unprocessed raw<xarray.DataArray (channel: 13, wavelength: 2, time: 23239)> Size: 5MB\n",
"<Quantity([[[0.227516 0.2297024 0.2261366 ... 0.2264519 0.2271665 0.226713 ]\n",
" [0.6354927 0.637668 0.6298023 ... 0.6072068 0.6087293 0.6091066]]\n",
"\n",
" [[0.1064704 0.1066212 0.1053444 ... 0.121114 0.1205022 0.1205441]\n",
" [0.2755033 0.2761615 0.2727006 ... 0.2911952 0.2900544 0.2909847]]\n",
"\n",
" [[0.5512474 0.5510672 0.5476283 ... 0.6179242 0.6188702 0.6187721]\n",
" [1.125532 1.1238331 1.1119423 ... 1.1817728 1.1819598 1.1832658]]\n",
"\n",
" ...\n",
"\n",
" [[0.3463254 0.3424951 0.3408207 ... 0.3929267 0.3941368 0.3945422]\n",
" [0.6978315 0.6875081 0.6857653 ... 0.7259991 0.7271688 0.7292138]]\n",
"\n",
" [[0.2225884 0.2187791 0.2195495 ... 0.2564863 0.2551258 0.2560233]\n",
" [0.3994258 0.3917637 0.389261 ... 0.4304597 0.430814 0.4331249]]\n",
"\n",
" [[0.2027881 0.1996586 0.2004866 ... 0.2318743 0.2311941 0.2330808]\n",
" [0.4666358 0.4554404 0.4561614 ... 0.4809749 0.4812827 0.4862896]]], 'volt')>\n",
"Coordinates:\n",
" * time (time) float64 186kB 0.0 0.128 0.256 ... 2.974e+03 2.974e+03\n",
" samples (time) int64 186kB 0 1 2 3 4 5 ... 23234 23235 23236 23237 23238\n",
" * channel (channel) object 104B 'S1D2' 'S1D3' 'S2D1' ... 'S7D7' 'S8D7'\n",
" source (channel) object 104B 'S1' 'S1' 'S2' 'S2' ... 'S6' 'S7' 'S8'\n",
" detector (channel) object 104B 'D2' 'D3' 'D1' 'D3' ... 'D7' 'D7' 'D7'\n",
" * wavelength (wavelength) float64 16B 760.0 850.0\n",
"Attributes:\n",
" data_type_group: unprocessed raw| [1] | Luke2021 | cedalion.data.get_fingertapping | Robert Luke and David McAlpine.\n", "fNIRS Finger Tapping Data in BIDS Format.\n", "September 2021.\n", "doi:10.5281/zenodo.5529797. |
| [2] | Tucker2022 | cedalion.io.snirf.read_snirf | Stephen Tucker, Jay Dubb, Sreekanth Kura, Alexander von Lühmann, Robert Franke, Jörn M. Horschig, Samuel Powell, Robert Oostenveld, Michael Lührs, Édouard Delaire, Zahra M. Aghajan, Hanseok Yun, Meryem A. Yücel, Qianqian Fang, Theodore J. Huppert, Blaise deB. Frederick, Luca Pollonini, David A. Boas, and Robert Luke.\n", "Introduction to the shared near infrared spectroscopy format.\n", "Neurophotonics, 10(1):013507, 2022.\n", "doi:10.1117/1.NPh.10.1.013507. |
| [3] | Delpy1988 | cedalion.nirs.cw.int2od | D. T. Delpy, M. Cope, P. van der Zee, S. Arridge, S. Wray, and J. Wyatt.\n", "Estimation of optical pathlength through tissue from direct time of flight measurement.\n", "Physics in Medicine and Biology, 33(12):1433–1442, 1988.\n", "doi:10.1088/0031-9155/33/12/008. |
| [4] | Villringer1997 | cedalion.nirs.cw.int2od | Arno Villringer and Britton Chance.\n", "Non-invasive optical spectroscopy and imaging of human brain function.\n", "Trends in Neurosciences, 20(10):435–442, 1997.\n", "doi:10.1016/S0166-2236(97)01132-6. |
| [5] | Huppert2009 | cedalion.sigproc.quality.mean_amp, cedalion.sigproc.quality.sd_dist, cedalion.sigproc.quality.snr | Theodore J. Huppert, Solomon G. Diamond, Maria A. Franceschini, and David A. Boas.\n", "Homer: a review of time-series analysis methods for near-infrared spectroscopy of the brain.\n", "Appl. Opt., 48(10):D280–D298, Apr 2009.\n", "doi:https://doi.org/10.1364/AO.48.00D280. |
| [6] | Pollonini2014 | cedalion.sigproc.quality.psp, cedalion.sigproc.quality.sci | Luca Pollonini, Cristen Olds, Homer Abaya, Heather Bortfeld, Michael S. Beauchamp, and John S. Oghalai.\n", "Auditory cortex activation to natural speech and simulated cochlear implant speech measured with functional near-infrared spectroscopy.\n", "Hearing Research, 309:84–93, 2014.\n", "doi:https://doi.org/10.1016/j.heares.2013.11.007. |
| [7] | Pollonini2016 | cedalion.sigproc.quality.psp, cedalion.sigproc.quality.sci | Luca Pollonini, Heather Bortfeld, and John S. Oghalai.\n", "PHOEBE: a method for real time mapping of optodes-scalp coupling in functional near-infrared spectroscopy.\n", "Biomedical Optics Express, 7(12):5104, Dec 2016.\n", "doi:10.1364/BOE.7.005104. |
| [8] | Sherafati2020 | cedalion.sigproc.quality._get_gvtd_threshold, cedalion.sigproc.quality.gvtd | Arefeh Sherafati, Abraham Z. Snyder, Adam T. Eggebrecht, Karla M. Bergonzi, Tracy M. Burns-Yocum, Heather M. Lugar, Silvina L. Ferradal, Amy Robichaux-Viehoever, Christopher D. Smyser, Ben J. Palanca, Tamara Hershey, and Joseph P. Culver.\n", "Global motion detection and censoring in high-density diffuse optical tomography.\n", "Human Brain Mapping, 41(14):4093–4112, 2020.\n", "doi:https://doi.org/10.1002/hbm.25111. |