2012/04/27
2012/04/19
Micromanaging Microarrays (Preview)
This past weekend, I spent a couple of days in Seattle reviewing my life as a postdoc there.
Part of that involved getting as many details on how my former lab used to process microarrays so that I could bring that experience to my new job. Basically, I want to bring what is currently a MS Excel and basic RMA affair into a more modern Systems Biology "Big Data" light.
Cue a couple suggestions from my buddy Chris Plaisier:
and the hard to implement but obvious advice of Environment Mapping (cataloging ALL genotypes and environment conditions for each microarray sample) the lot of existing experiments pronto.
As I've volunteered to present these concepts in about two weeks time, next week is going to be fun.
Part of that involved getting as many details on how my former lab used to process microarrays so that I could bring that experience to my new job. Basically, I want to bring what is currently a MS Excel and basic RMA affair into a more modern Systems Biology "Big Data" light.
Cue a couple suggestions from my buddy Chris Plaisier:
and the hard to implement but obvious advice of Environment Mapping (cataloging ALL genotypes and environment conditions for each microarray sample) the lot of existing experiments pronto.
As I've volunteered to present these concepts in about two weeks time, next week is going to be fun.
2012/04/13
One app, three languages
This past week at work I had the opportunity to code the same algorithm using each of the three scientific programming/scripting languages I'm familiar with:
The list above is the order that the (re)-coding was done and serves as a beginning of an answer as to why I had|wanted to do such repetitive work.
Before getting into the details first the problem: Grab multiple optimized DNA sequences in a MS Excel workbook and format them as a FASTA text file for use with a webapp for rare codon analysis. Prior to seeking my help, users were manually copying the sequences (located in one cell across multiple sheets) into a MS Word document. This was fine for 2-5 sequences, but got seriously tedious and error prone for anything >10.
Lastly, this is all done in Windows (for added craziness).
xlsfinfo
xlsread
xlswrite
Adding the (also expensive) Bioinformatics Toolbox gives FASTA file io via
fastaread
fastawrite
Using the Matlab Compiler (again, if you've paid for it) this distills nicely into a tidy command line executable.
The problems began (as usual) immediately after deployment.
First, in order to run any compiled Matlab code, users need to install the weighty (~400MB) Matlab Component Runtime (MCR), which in corporate IT-lockdown land is it's own form of enjoyment.
Second, and a horrendous PITA if you ask me, the version of the MCR users need depends on the version of Matlab the code was compiled in. Worse, there is no backward compatibility. In this case, users needed version 7.16 of the runtime to correspond with my Matlab 2011b. However, the program that generated the sequences in the first place (also a Matlab compile job) was made with Matlab 2009a.
It was late in the afternoon, the IT guys were gone, and I didn't want to have to deal with any craziness of conflicting runtimes.
Sorry Matlab, you suck.
MS Excel spelunking:
xlrd
xlwt
FASTA file acrobatics (and much much more):
BioPython
As an important note, the python code completes almost instantaneously whereas the Matlab code took at least 5 seconds (about 1 sec per worksheet in the source .xlsx file). I don't know why Matlab takes so long to get/put data from/into an Excel workbook, but I sure hope the Mathworks engineers are working on it. Sure, this is slightly unfair since python is byte-compiled when run, but on a modern PC with 1GHz of multi-core processing power and 3GB of RAM, I expect performance, darn-it.
As a small slight to the Python camp, the xlrd/xlwt modules are only compatible with the older .xls (Microsoft Excel 97-2000) format files and not the newer XML based .xlsx files. So it does require one extra step ... par for the course.
Compiling to an console executable is made easy with Py2Exe.
Deploying is a snap - zip and email everything in the ./dist folder of where you Py2Exe'd your source code.
Of course, getting users that were originally in happy point and click land to work at the C: prompt kinda stopped this awesome train of free and open source progress dead in its tracks.
xlsx
FASTA files, Bioinformatics and statistics go hand-in-hand, this is pretty much a given:
seqinr
As far as speed goes, the R code bested Matlab and was on par with Python. Pretty interesting and consistent with the benchmarks posted by the Julia team.
A small word of warning, the xlsx package uses the Apache POI java library in the background and does run into significant memory cloggery (at least on my workstation) when working with sheets heavily laden with rows and/or columns.
Compiling, well, I'm not completely sure it exists yet (although RCC looks interesting). Of course, who needs to compile if you can just drop this on your webserver behind rApache and a simple webform. There, user command-line aversion solved.
The list above is the order that the (re)-coding was done and serves as a beginning of an answer as to why I had|wanted to do such repetitive work.
Before getting into the details first the problem: Grab multiple optimized DNA sequences in a MS Excel workbook and format them as a FASTA text file for use with a webapp for rare codon analysis. Prior to seeking my help, users were manually copying the sequences (located in one cell across multiple sheets) into a MS Word document. This was fine for 2-5 sequences, but got seriously tedious and error prone for anything >10.
Lastly, this is all done in Windows (for added craziness).
Round 1: Matlab (aka the 500lb gorilla)
Out of the (very expensive) box it has MS Excel read/write capabilities via the functionsxlsfinfo
xlsread
xlswrite
Adding the (also expensive) Bioinformatics Toolbox gives FASTA file io via
fastaread
fastawrite
Code:
Using the Matlab Compiler (again, if you've paid for it) this distills nicely into a tidy command line executable.
The problems began (as usual) immediately after deployment.
First, in order to run any compiled Matlab code, users need to install the weighty (~400MB) Matlab Component Runtime (MCR), which in corporate IT-lockdown land is it's own form of enjoyment.
Second, and a horrendous PITA if you ask me, the version of the MCR users need depends on the version of Matlab the code was compiled in. Worse, there is no backward compatibility. In this case, users needed version 7.16 of the runtime to correspond with my Matlab 2011b. However, the program that generated the sequences in the first place (also a Matlab compile job) was made with Matlab 2009a.
It was late in the afternoon, the IT guys were gone, and I didn't want to have to deal with any craziness of conflicting runtimes.
Sorry Matlab, you suck.
Round 2: Python (parsel tongue any one?)
There's a lot of hubub about how NumPy/SciPy + matplotlib in distributions like Python(X,Y) can totally replace Matlab - FOR FREE! I've yet to really delve into that module stack. For the task at hand, bare python has all that's needed with a few modules (again ALL FREE)MS Excel spelunking:
xlrd
xlwt
FASTA file acrobatics (and much much more):
BioPython
Code:
As an important note, the python code completes almost instantaneously whereas the Matlab code took at least 5 seconds (about 1 sec per worksheet in the source .xlsx file). I don't know why Matlab takes so long to get/put data from/into an Excel workbook, but I sure hope the Mathworks engineers are working on it. Sure, this is slightly unfair since python is byte-compiled when run, but on a modern PC with 1GHz of multi-core processing power and 3GB of RAM, I expect performance, darn-it.
As a small slight to the Python camp, the xlrd/xlwt modules are only compatible with the older .xls (Microsoft Excel 97-2000) format files and not the newer XML based .xlsx files. So it does require one extra step ... par for the course.
Compiling to an console executable is made easy with Py2Exe.
Deploying is a snap - zip and email everything in the ./dist folder of where you Py2Exe'd your source code.
Of course, getting users that were originally in happy point and click land to work at the C: prompt kinda stopped this awesome train of free and open source progress dead in its tracks.
Round 3: R (statistics will help you find that buried treasure, er p-value)
R directly works with MS Excel files? Yup:xlsx
FASTA files, Bioinformatics and statistics go hand-in-hand, this is pretty much a given:
seqinr
Code:
As far as speed goes, the R code bested Matlab and was on par with Python. Pretty interesting and consistent with the benchmarks posted by the Julia team.
A small word of warning, the xlsx package uses the Apache POI java library in the background and does run into significant memory cloggery (at least on my workstation) when working with sheets heavily laden with rows and/or columns.
Compiling, well, I'm not completely sure it exists yet (although RCC looks interesting). Of course, who needs to compile if you can just drop this on your webserver behind rApache and a simple webform. There, user command-line aversion solved.
Wrap-up
So what did this exercise in redundancy teach me? Well, thanks to the plethora of open-source tools, there is more than one way to skin a software deployment cat. It also shows how completely originally niche platforms like Python and R have come to parity with juggernauts like Matlab. Last, it has me satisfied (for now) in my programming poly-glottery.2011/05/22
MLINE for Control Channel Routing
After installing AutoCAD 2012 today I came across the MLINE command. It is an easy way to draw a set of parallel lines that follow a path. Steps:
- MLINE at the command prompt
- Enter J to choose justification - Zero seems most appropriate: centers the path taken between the lines drawn.
- Default scale is 1, which is 1:1 with the current drawing units. Set this to be the width of your control channel.
- Start tracing a path point by point.
- When finished hit ESC.
- EXPLODE the MLINE you've just drawn
- Use PEDIT to Join all the resultant line segments
(defun c:m2p (/ vlst ovar ent ss elast)
(setq vlst '("CMDECHO" "PEDITACCEPT")
ovar (mapcar 'getvar vlst))
(if (and (setq ent (car (entsel "\nSelect Multi-Line...")))
(eq "MLINE" (cdadr (entget ent))))
(progn
(mapcar 'setvar vlst '(0 1))
(command "_explode" ent) (setq ss (ssadd))
(mapcar '(lambda (x) (ssadd x ss)) (Ent_List_to_End ent))
(setq elast (entlast))
(command "_pedit" "_M" ent ss elast "" "_J" "" ""))
(princ "\n No Multi-Line Selected "))
(mapcar 'setvar vlst ovar)
(princ))
(defun Ent_List_to_End(ent / a)
(reverse
(if(setq a(entnext ent))
(cons ent(Ent_List_to_End a)))))
2010/02/11
Breathing moisture barrier film and bottom-less plates
(More stuff out of my "ideas" Google Doc)
ACLAR film:
Typically used for embedding histology sections. Fluorcarbon based, clear as glass, flexible, and easy to cut - I'm assuming doesn't produce shards. According to the specs, it has a high O2 permeability, but low to nill moisture transmission.
Typically used for embedding histology sections. Fluorcarbon based, clear as glass, flexible, and easy to cut - I'm assuming doesn't produce shards. According to the specs, it has a high O2 permeability, but low to nill moisture transmission.
Potentially useful for a number of things:
These would be useful if the LED tray experiments get started again - no need to custom manufacture the base plate.- a way to get around special rehydration channels or fluorinert in valve lines in PDMS based microfluidics
- top
- thick casting for structural support
- aclar film
- control layer (thin)
- flow layer (thin)
- glass slide/coverslip
- bottom
- a lid for microplate readers - I wonder if they make the stuff with adhesive backing.
These might also be useful for more standardized microfluidic assembly - e.g. select a large bottomless well plate (6-well). Glue a chip (glass and all) to one of the wells from the bottom.
Interesting references
These have been languishing in a Google Doc for some time (Aug 2009 to be exact). Perhaps having them here might help me to read them more carefully ...
Oh, hell yes ...
Influence of Heavy Metals on Microbial Growth Kinetics Including Lag Time: Mathematical Modeling and Experimental Verification.
PMID: 19496634. This looks like an interesting model to study - esp given the specific ways Zn and Cu affect halo growth
Also,
Removal of multi-heavy metals using biogenic manganese oxides generated by a deep-sea sedimentary bacterium - Brachybacterium sp. strain Mn32.
PMID: 19383675. Apparently oxides of Mn(II) can sequester Zn, maybe copper. Check to see how Mn(II) specific resistance is differentially expression during Zn/Cu stress.
Oh, hell yes ...
A P-type ATPase importer that discriminates between essential and toxic transition metals.
PMID: 19264958. A heavy metal importer with specificity to Cu and Zn is identified in Pseudomonas aeruginosa Q9I147. There are several importers described or referenced here with specificities only for Cu or Zn. Get protein sequences and blast against the Halo proteome.Last, worthwhile to review. I might have this one already
The role of efflux in bacterial resistance to soft metals and metalloids.
PMID: 107301852009/12/28
Milli (Vanilli) Fluidics - Part 2
4% agarose in PBS (200 g/L salinity, pH 7.2)
The idea has been refined:
Making grooves
Attempt 1
Used two fine gauge wires covered in tape. Found some wire in the common tool area, from the looks of it, it was 30awg. The wires provided the depth for the channels while the tape smoothed out the profile and made the weir in between the wires/channels.
The wires were positioned so that neither channel extended completely across the pad region, but with a small (~5-10mm) of overlapping region for the weir.
Wire+tape 'mold' was assembled on a regular office grade transparency sheet to make release of the mold from the agarose pad easier.
Result
Channels made this way were really really deep. The pad patterned was also too small. so channels went all the way across. Didn't test loading with cells. Regular Scotch tape (matte finish) produced a lot of observable roughness in the resultant patterned pad. Prefer it to be smooth.
Attempt 2
Instead of wire, used strips of tape, covered in tape. Much easier to position. Used clear packing tape as the 'smoothing' layer.
Result
Grooves pattern probably caused by scratches from tweezers used to smooth out air bubbles. Cells get caught in these grooves. Cells dropped onto the pad (5uL) before coverslip. Trapping in the weir doesn't seem robust.
Attempt 3
Discontinuous single channel patterned using wires heavily smoothed with tape.
Result
Attempted cell loading via channel. Appears that there is enough non-uniformity (non-flatness) in the pad that promotes bulk capillary action across the entire pad.
Attempt 4
Fine grit sand paper roughened packing tape. Sandpaper drawn over surface once with moderate pressure, to create deep parallel grooves (will result in channel walls on pad).
Result
The pattern was evident when the pad was dry, but once wetted, it was not observable. Only 2uL of cells spotted (small pad). Cells in center popped - probably from over pressure of coverslip.
Todo: Attempt 5
PDMS 'mold box'. Put a small transparency film roughened with sandpaper on a flat surface. Outline the film with tape, approx 2-5mm from the film edge. Cast PDMS over the top of this.
The PDMS will be come a small molding box to create a pad the size of the transparancy film with grooves from the sandpaper roughened area. Drop this box over a spot of molten agarose on a glass slide. Cool the assembly and peel away (hopefully with the agarose portion still stuck to the glass slide).
The idea has been refined:
- Make two adjacent channels with a weir in between
- Cells get caught in the weir as flow moves in from one channel, over the weir, and out the other channel
- Same flow path is used for adding stimulant
Making grooves
Attempt 1
Used two fine gauge wires covered in tape. Found some wire in the common tool area, from the looks of it, it was 30awg. The wires provided the depth for the channels while the tape smoothed out the profile and made the weir in between the wires/channels.
The wires were positioned so that neither channel extended completely across the pad region, but with a small (~5-10mm) of overlapping region for the weir.
Wire+tape 'mold' was assembled on a regular office grade transparency sheet to make release of the mold from the agarose pad easier.
Result
Channels made this way were really really deep. The pad patterned was also too small. so channels went all the way across. Didn't test loading with cells. Regular Scotch tape (matte finish) produced a lot of observable roughness in the resultant patterned pad. Prefer it to be smooth.
Attempt 2
Instead of wire, used strips of tape, covered in tape. Much easier to position. Used clear packing tape as the 'smoothing' layer.
Result
Grooves pattern probably caused by scratches from tweezers used to smooth out air bubbles. Cells get caught in these grooves. Cells dropped onto the pad (5uL) before coverslip. Trapping in the weir doesn't seem robust.
Attempt 3
Discontinuous single channel patterned using wires heavily smoothed with tape.
Result
Attempted cell loading via channel. Appears that there is enough non-uniformity (non-flatness) in the pad that promotes bulk capillary action across the entire pad.
Attempt 4
Fine grit sand paper roughened packing tape. Sandpaper drawn over surface once with moderate pressure, to create deep parallel grooves (will result in channel walls on pad).
Result
The pattern was evident when the pad was dry, but once wetted, it was not observable. Only 2uL of cells spotted (small pad). Cells in center popped - probably from over pressure of coverslip.
Todo: Attempt 5
PDMS 'mold box'. Put a small transparency film roughened with sandpaper on a flat surface. Outline the film with tape, approx 2-5mm from the film edge. Cast PDMS over the top of this.
The PDMS will be come a small molding box to create a pad the size of the transparancy film with grooves from the sandpaper roughened area. Drop this box over a spot of molten agarose on a glass slide. Cool the assembly and peel away (hopefully with the agarose portion still stuck to the glass slide).
Subscribe to:
Posts (Atom)