Skip to content

Commit 910f54d

Browse files
authored
Merge pull request #32 from Tauffer-Consulting/compiled-waveclus
Added waveclus-compiled
2 parents 4f928bb + 0a0b689 commit 910f54d

File tree

5 files changed

+246
-0
lines changed

5 files changed

+246
-0
lines changed

waveclus-compiled/Dockerfile

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
FROM waveclus-matlab-base
2+
3+
RUN chmod 755 /usr/bin/mlrtapp/waveclus_compiled
4+
ENV PATH="/usr/bin/mlrtapp:${PATH}"
5+
6+
RUN apt-get update -y
7+
RUN apt-get install software-properties-common -y
8+
RUN add-apt-repository ppa:deadsnakes/ppa -y
9+
RUN apt-get install git python3.8 python3.8-dev -y
10+
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
11+
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 2
12+
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
13+
RUN apt-get install python3-pip -y
14+
RUN update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1
15+
RUN pip install -U pip
16+

waveclus-compiled/README.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Waveclus Compiled Docker Images
2+
3+
This documentation is intended to show how to create a Docker image with Matlab compiled Waveclus sorter. The main goal of this project is to avoid the requirement of Matlab Licenses and also abstract the installation and setup steps to run the sorter
4+
5+
There are three main steps for generating a functional Waveclus docker image:
6+
1. Compile Waveclus as Matlab's Standalone Application
7+
2. Create a (base) docker image with Matlab Runtime and the compiled application from step 1
8+
3. Extend the docker image from step 2 for improvements and fixes
9+
10+
## Requirements
11+
- Packaging a MATLAB Docker image is supported on Linux only
12+
- Docker
13+
- Matlab
14+
15+
### Matlab Requirements
16+
- MATLAB Compiler
17+
- Signal Processing Toolbox
18+
- Statistics and Machine Learning Toolbox
19+
- Wavelet Toolbox
20+
21+
## Compiling Waveclus as Matlab's Standalone Application
22+
- Git clone or Download waveclus [source code](https://github.com/csn-le/wave_clus)
23+
- Open Matlab
24+
- Set Matlab's workspace folder to `/path/to/spikeinterface-dockerfiles/waveclus-compiled`
25+
- Run `mcc` command with `waveclus_compiled.m` as base and adding path to git cloned waveclus:
26+
```matlab
27+
>> mcc -m waveclus_compiled.m -a <git-cloned-path>/wave_clus
28+
```
29+
30+
## Generating Base Docker Image
31+
- To generate the base docker image (called `waveclus-matlab-base`) with the compiled application, run the following command in Matlab console:
32+
```matlab
33+
>> compiler.package.docker('waveclus_compiled', 'requiredMCRProducts.txt', 'ImageName', 'waveclus-matlab-base')
34+
```
35+
36+
37+
- [Optional] Files generated by Matlab Compiler can be deleted:
38+
- In your terminal, go to the `waveclus-compiled` folder in this project:
39+
```bash
40+
$ cd /path/to/spikeinterface-dockerfiles/waveclus-compiled
41+
```
42+
43+
- Run `rm` command:
44+
```bash
45+
$ rm -r \
46+
includedSupportPackages.txt \
47+
waveclus-matlab-basedocker/ \
48+
waveclus_compiled \
49+
mccExcludedFiles.log \
50+
readme.txt \
51+
requiredMCRProducts.txt \
52+
run_waveclus_compiled.sh \
53+
unresolvedSymbols.txt
54+
```
55+
56+
## Extending Base Image and creating final image
57+
The Dockerfile in this folder applies some fixes and updates to the base image generated automatically by Matlab in order to properly run waveclus:
58+
59+
60+
- In your terminal, go to the folder for this project:
61+
```
62+
$ cd /path/to/spikeinterface-dockerfiles/waveclus-compiled
63+
```
64+
65+
- Run build script:
66+
```
67+
$ source build.sh
68+
```

waveclus-compiled/build.sh

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/bin/bash
2+
3+
docker build -t spikeinterface/waveclus-compiled-base:latest -t spikeinterface/waveclus-compiled-base:0.1.0 .
4+

waveclus-compiled/push.sh

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/bin/bash
2+
3+
docker push --all-tags spikeinterface/waveclus-compiled-base
4+
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
function waveclus_compiled(outputFolder)
2+
3+
% par.mat file should contain a variable called par_input
4+
load(fullfile(outputFolder, 'par_input.mat'));
5+
6+
% Default parameters
7+
S_par = set_parameters_ss();
8+
9+
% Update with custom parameters
10+
S_par = update_parameters(S_par, par_input, 'relevant');
11+
12+
try
13+
cd(outputFolder);
14+
nch = 0;
15+
while true
16+
aux_filename = fullfile(outputFolder, ['raw' int2str(nch+1) '.h5']);
17+
if exist(aux_filename,'file')
18+
nch = nch +1;
19+
vcFile_mat{nch} = aux_filename;
20+
else
21+
break
22+
end
23+
end
24+
if nch==1
25+
% Run waveclus batch mode.
26+
Get_spikes(vcFile_mat{1}, 'par', S_par);
27+
vcFile_spikes = strrep(vcFile_mat{1}, '.h5', '_spikes.mat');
28+
Do_clustering(vcFile_spikes, 'make_plots', false,'save_spikes',false);
29+
[vcDir_, vcFile_, ~] = fileparts(vcFile_mat{1});
30+
vcFile_cluster = fullfile(vcDir_, ['times_', vcFile_, '.mat']);
31+
else
32+
% Run waveclus batch mode.
33+
pol_file = fopen('polytrode1.txt','w');
34+
cellfun(@(x) fprintf(pol_file ,'%s\n',x),vcFile_mat);
35+
fclose(pol_file);
36+
Get_spikes_pol(1, 'par', S_par);
37+
vcFile_spikes = 'polytrode1_spikes.mat';
38+
Do_clustering(vcFile_spikes, 'make_plots', false,'save_spikes',false);
39+
[vcDir_, ~, ~] = fileparts(vcFile_mat{1});
40+
vcFile_cluster = fullfile(vcDir_, ['times_polytrode1', '.mat']);
41+
end
42+
newfile = fullfile(vcDir_, 'times_results.mat');
43+
if exist(vcFile_cluster,'file')
44+
movefile(vcFile_cluster,newfile,'f');
45+
else
46+
load(vcFile_spikes,'par');
47+
par = update_parameters(S_par, par, 'relevant');
48+
cluster_class = zeros(0,2);
49+
save(newfile,'cluster_class','par');
50+
end
51+
catch
52+
fprintf('----------------------------------------');
53+
fprintf(lasterr());
54+
quit(1);
55+
end
56+
quit(0);
57+
end
58+
59+
60+
function par = set_parameters_ss()
61+
62+
% LOAD PARAMS
63+
par.segments_length = 5; % length (in minutes) of segments in which the data is cut (default 5min).
64+
65+
66+
% PLOTTING PARAMETERS
67+
par.cont_segment = true;
68+
par.max_spikes_plot = 1000; % max. number of spikes to be plotted
69+
par.print2file = true; % If is not true, print the figure (only for batch scripts).
70+
par.cont_plot_samples = 100000; % number of samples used in the one-minute (maximum) sample of continuous data to plot.
71+
par.to_plot_std = 1; % # of std from mean to plot
72+
par.all_classes_ax = 'mean'; % 'mean'/'all'. If it's 'mean' only the mean waveforms will be plotted in the axes with all the classes
73+
par.plot_feature_stats = false;
74+
75+
76+
% SPC PARAMETERS
77+
par.mintemp = 0.00; % minimum temperature for SPC
78+
par.maxtemp = 0.251; % maximum temperature for SPC
79+
par.tempstep = 0.01; % temperature steps
80+
par.SWCycles = 100; % SPC iterations for each temperature (default 100)
81+
par.KNearNeighb = 11; % number of nearest neighbors for SPC
82+
par.min_clus = 20; % minimum size of a cluster (default 60)
83+
par.max_clus = 200; % maximum number of clusters allowed (default 200)
84+
par.randomseed = 0; % if 0, random seed is taken as the clock value (default 0)
85+
%par.randomseed = 147; % If not 0, random seed
86+
%par.temp_plot = 'lin'; % temperature plot in linear scale
87+
par.temp_plot = 'log'; % temperature plot in log scale
88+
89+
par.c_ov = 0.7; % Overlapping coefficient to use for the inclusion criterion.
90+
par.elbow_min = 0.4; %Thr_border parameter for regime border detection.
91+
92+
% DETECTION PARAMETERS
93+
par.tmax = 'all'; % maximum time to load
94+
%par.tmax= 180; % maximum time to load (in sec)
95+
par.tmin= 0; % starting time for loading (in sec)
96+
par.w_pre = 20; % number of pre-event data points stored (default 20)
97+
par.w_post = 44; % number of post-event data points stored (default 44)
98+
par.alignment_window = 10; % number of points around the sample expected to be the maximum
99+
par.stdmin = 5; % minimum threshold for detection
100+
par.stdmax = 50; % maximum threshold for detection
101+
par.detect_fmin = 300; % high pass filter for detection
102+
par.detect_fmax = 3000; % low pass filter for detection (default 1000)
103+
par.detect_order = 4; % filter order for detection
104+
par.sort_fmin = 300; % high pass filter for sorting
105+
par.sort_fmax = 3000; % low pass filter for sorting (default 3000)
106+
par.sort_order = 2; % filter order for sorting
107+
par.ref_ms = 1.5; % detector dead time, minimum refractory period (in ms)
108+
par.detection = 'pos'; % type of threshold ('pos','neg','both')
109+
% par.detection = 'neg';
110+
% par.detection = 'both';
111+
112+
% INTERPOLATION PARAMETERS
113+
par.int_factor = 5; % interpolation factor
114+
par.interpolation = 'y'; % interpolation with cubic splines (default)
115+
% par.interpolation = 'n';
116+
117+
118+
% FEATURES PARAMETERS
119+
par.min_inputs = 10; % number of inputs to the clustering
120+
par.max_inputs = 0.75; % number of inputs to the clustering. if < 1 it will the that proportion of the maximum.
121+
par.scales = 4; % number of scales for the wavelet decomposition
122+
par.features = 'wav'; % type of feature ('wav' or 'pca')
123+
%par.features = 'pca'
124+
125+
126+
% FORCE MEMBERSHIP PARAMETERS
127+
par.template_sdnum = 3; % max radius of cluster in std devs.
128+
par.template_k = 10; % # of nearest neighbors
129+
par.template_k_min = 10; % min # of nn for vote
130+
%par.template_type = 'mahal'; % nn, center, ml, mahal
131+
par.template_type = 'center'; % nn, center, ml, mahal
132+
par.force_feature = 'spk'; % feature use for forcing (whole spike shape)
133+
%par.force_feature = 'wav'; % feature use for forcing (wavelet coefficients).
134+
par.force_auto = true; %automatically force membership (only for batch scripts).
135+
136+
% TEMPLATE MATCHING
137+
par.match = 'y'; % for template matching
138+
%par.match = 'n'; % for no template matching
139+
par.max_spk = 40000; % max. # of spikes before starting templ. match.
140+
par.permut = 'y'; % for selection of random 'par.max_spk' spikes before starting templ. match.
141+
% par.permut = 'n'; % for selection of the first 'par.max_spk' spikes before starting templ. match.
142+
143+
% HISTOGRAM PARAMETERS
144+
par.nbins = 100; % # of bins for the ISI histograms
145+
par.bin_step = 1; % percentage number of bins to plot
146+
end
147+
148+
149+
% Overwriting supported_wc_extensions function from waveclus source
150+
function formats = supported_wc_extensions()
151+
% Return a cell of arrays with the formats supported/needed for spikeinterface.
152+
formats = {'h5', 'mat'};
153+
end
154+

0 commit comments

Comments
 (0)