DemultiplexSource

Data Stream Processing > Processing Modules > General manipulation Modules

Description

The DemultiplexSource processing module is designed to de-multiplex barcoded data like 10x genomics single-cell data. The demultiplex key can be anything from a cellID or unique-molecule-ID or any future multiplexing barcode method. At runtime this module will create sub-datasources within the primary DataSource effectively demultiplexing the barcodes from either BAM files or custom processed files (like single-cell CAGE ctss bed files). This makes the loading of single-cell data much easier and efficient since the cell demultiplexing does not have to be performed prior to load generating 10000s of extra files.

Parameters

• <source_mode> : defines which data-source the demultiplexing is applied to. Possible values are:
  • featuresource : create sub-sources on the primary FeatureSource
  • experiment : create demux sub-sources on the corresponding Experiment
• <demux_mdkeys> : the metadata key for the Feature metadata which encode the demultiplexing ID.
• <side_linking_mdkey> : if merging demux metadata from side_stream, use metadata column type for linking to demux_mdkeys
• <full_demux> : create all demux sources based on the side_stream metadata
• <demux_source_mdkey> : append the metadata for the source metadata key to the demux_key. This allows for creating unique demux-keys from merged files to allow for the side_stream linking of metadata.

Example

This script combines DemultiplexSource with FeatureEmitter / TemplateCluster to process single-cell CAGE data as signal-histogram visualization. For this example the single-cell CAGE data was loaded as a BED6 CTSS file where the bed.name column has the cellID and the score column has the CTSS count. Dynamic metadata is linked to the newly demuxed subsources by using a side_stream with an uploaded metadata file and using the CellID column from the metadata file to link with the modules generated demux_mdkey based on the BED6 eedb:name.

<zenbu_script>
	<datastream name="cell_mdata" output="full_feature">
		<source id="B9DECA55-F95C-447E-818E-BB2B25291DBF::1:::FeatureSource"/>
	</datastream>
	<stream_processing>
		<spstream module="DemultiplexSource">
			<source_mode>experiment</source_mode>
			<demux_mdkeys>eedb:name</demux_mdkeys>
			<side_linking_mdkey>CellID</side_linking_mdkey>
			<side_stream>
				<spstream module="Proxy" name="cell_mdata"/>
			</side_stream>
		</spstream>
		<spstream module="MetadataFilter">
			<mdata_mode>experiment</mdata_mode>
			<inverse>false</inverse>
			<mdata type="Cluster"/>
		</spstream>
		<spstream module="TemplateCluster">
			<overlap_mode>5end</overlap_mode>
			<expression_mode>sum</expression_mode>
			<ignore_strand>false</ignore_strand>
			<overlap_subfeatures>false</overlap_subfeatures>
			<side_stream>
				<spstream module="FeatureEmitter">
					<fixed_grid>true</fixed_grid>
					<both_strands>true</both_strands>
				</spstream>
			</side_stream>
		</spstream>
	</stream_processing>
</zenbu_script>

Example of a linked metadata file. The linking-key should match the internal attribute from the source file. For common single-cell BAM files, the attribute is stored as CB in the BAM alignment aux-data columns. Create this metadata file separately as a simple tab-text OSCtable and load into ZENBU. After loading find the internal ZENBU ID for that uploaded database and use that as a <datastream><source> in your script.

CellID	Cluster	Lib.size
AAACCTGTCGCAAACT-1	0	38374
AAACGGGAGCTTCGCG-1	3	6966
AAAGATGGTCAAGCGA-1	1	6675

Example2

Here is an example using two single cell BAM files and a merged metadata file for both experiments. In this case the two BAM files correspond to two different WildType mouse conditions. To avoid cell-barcode collisions the metadata uses an extended barcode with the sampleID appended. To create this demux-key with the module, we must use a combinattion of CB from each BAM alignment aux-data <demux_mdkeys>CB</demux_mdkeys>, and sam:sample from the BAM header <demux_source_mdkey>sam:sample</demux_source_mdkey>

<zenbu_script>
	<datastream name="cell_mdata" output="full_feature">
		<source id="695E018B-3FEC-46FB-92F7-177B2AE0CEF0::1:::FeatureSource"/>
	</datastream>
	<stream_processing>
		<spstream module="DemultiplexSource">
			<source_mode>experiment</source_mode>
			<demux_mdkeys>CB</demux_mdkeys>
			<full_demux>true</full_demux>
			<demux_source_mdkey>sam:sample</demux_source_mdkey>
			<side_linking_mdkey>cell_barcode</side_linking_mdkey>
			<side_stream>
				<spstream module="Proxy" name="cell_mdata"/>
			</side_stream>
		</spstream>
		<spstream module="TemplateCluster">
			<overlap_mode>height</overlap_mode>
			<expression_mode>sum</expression_mode>
			<ignore_strand>false</ignore_strand>
			<overlap_subfeatures>true</overlap_subfeatures>
			<side_stream>
				<spstream module="FeatureEmitter">
					<fixed_grid>true</fixed_grid>
					<both_strands>true</both_strands>
				</spstream>
			</side_stream>
		</spstream>
	</stream_processing>
</zenbu_script>