Testing. Where’s the best place to add value?

Until every last human (and animal) susceptible to SARS-CoV-2 is safe we are not safe. That could happen through multiple routes of vaccines, anti-virals, or testing & contact tracing. Natural immunity will be the only option for some of our poorest but it’s not clear how much protection that will afford in the long run, particularly when novel strains emerge.

This post is going to focus on testing. On the options facing UK / European / US compared with other countries with even less testing capacity and reliable infrastructure.

As I posted previously, the obvious holy grail of testing would be something 90% of citizens could do at home before they leave in the morning.

But let’s start from where we are now and we’ll figure out how far we can get towards our goal.


Getting accurate, affordable “infectiousness” testing for as many people as possible, i.e. focusing on preventative testing to enable effective quarantine.

Strategy [updated 2020-05-14]

3 strands to explore, 1 “completed”

  • “Completed” – encourage existing tech to be opened and or widely licensed, e.g. OpenCovidPledge.org
  • “In progress” – Talk about existing open source tech, like Bomb.Bio and OpenCell e.g.
    • assisting them publish “fast data” validation papers
    • talk to existing pathology lab managers about their shortages of RNA purification reagents
    • https://testingmethods.crowdicity.com/post/3163640 and given opportunity to present on NHS innovations call all the labs using it.
  • “In progress” – Bigger picture: help unlock creativity, industrial and clinical participation from a much wider range of actors:
    • Continuous validation techniques
    • Clinical sample standards
  • “In progress” – reference implementation / cheap testing options
    • facilitate the competitive production of open-source designs for decentralised high throughput automated RT-qPCR testing
    • need the end-to-end view: coronaopensource.github.io


UK [Correct as of 2020-05-14]

In the UK we’ve still got some of our NHS labs short of RNA extraction & purification chemicals [private communications]. They can’t work. We’ve got the mega labs and they will get up to capacity of 100k per day but not there yet. I suspect the mega labs will get solved within a week if it’s coordination or a month if it’s supply of swabs etc. The former problem of smaller labs missing reagents I don’t know but again it’s going to be weeks to months and the large biotechs will supply them.

Open Science: Bomb.Bio, OpenCell, Crick institute

OpenCell have been waving the flag since mid January, a full 45 days before I woke up and about 60 days before most people in the UK. Well done. It would explain why they’re ready to start validating their fully open-source system on real samples.

The Crick have also been profoundly useful with their protocols.

What’s the next step?

I think we need to be thinking about the big picture. So I’m going to plan it out. What it needs at what point.

An attempt at a detailed national testing “plan”

This is my attempt to make sense (from the outside) of all the hard work thousands of people are doing and the hard science and options the lab scientists, managers, technicians, suppliers are making available on a daily basis.


My diary

For my benefit to start keeping track of what’s happening.


Took apart pipette to see how they work. In someways really simple.


Talked to NHS about adding Bomb.bio and OpenCell to path lab innovations call.
Shared Armeevent progress with HelpfulEngineering.
Asked DoctorCareAnywhere for tech support integrating records into NHS.
Drafted email re patent infringement for RNA purification.
09:30 Remember I hadn’t put calls with Public Health Wales and UK NHS path labs on list of things done… RNA purification now being taken forwards so advice from Rich Hopes that it’ll work its way through was correct as it now on Pathology Alliance NHS Engagement Webinar.
Contacted path lab re RNA precipitating on swab and how to increase sensitivity.
Reviewed superficially genius but actually unfortunately not acceptable binary search approach to testing: https://testingmethods.crowdicity.com/post/3173770
Assess chemical reagents for RT-qPCR step.
Thought about and rejected again microfluidics approach due to small volumes.
Finished assessing OpenCell / Bomb.bio / Crick reagent costs. Seems to be £1 per test which is about x10 lower than I was expecting so I am pretty confused. Must have made a mistake.


Sync up with __ and discuss challenges UK faces for testing.
There are NHS labs with skilled staff and machines but no reagents, they don’t have access to (university) labs that can make up the reagents they need.
There are labs with machines but none of the special plastics (with reagents preloaded).
We can’t give them an end to end machine because it needs to be validated… ?by UKAS?
Talked to more (amazing) NHS lab pathology managers (I am so grateful to all those working hard in the NHS and else where). Said they’re still reporting shortages of swabs. A lot of labs are now fine but not being run at capacity yet.
Structure data around end to end testing protocols
Thank Bomb.bio team including those from NZ who stayed up until 2 am


OpenCell write up starts 1 day after they start getting validation results using Bomb.bio automated on OpenTrons. Pass v rough half draft to them.
Feedback to Armeevent on possible strategies.
Continue structuring data around end to end testing protocols.
Start putting data into structure.


Feedback to Hospital mask fit project.
Continue structuring data around end to end testing protocols.
Chat with collective intelligence project lead.
Updating SARS-CoV-2 sample location post.




Talk with OpenCell regarding write up
Schedule a call regarding OpenCell tech
Reading FDA EUA policy
More testingmethods crowdicity posts


Post testingmethods crowdicity sample tube standardisation idea
Read SystemShiftLab proposal
Send message re “Test the testers”
Finish skim reading FDA EUA policy
Publish “Continuous Validation


Got feedback on Continuous Validation from Charlotte, Ian
Image for Continuous Validation
Send to get feedback from Catherine and CofCI


UK newspaper headlines about easing lockdown despite number of cases remaining flat (though test per positive gone up to ~15) makes me anxious.
Read the UK guidance again on how to help provide testing.
Offer to represent Bomb.Bio protocol on UK pathology lab call
Read through comments on Continuous Validation paper from Karthik and Daniel.
Respond to comments.
Think about sample standards. Think about other ways to validate labs quickly.
Look up some UEA on internal references used.


(Fix washing machine)
Put in controls data for 28/48 of testing map EUA SARS-CoV-2 tests
Long talk with and calm kind guidance from Karthik


Try using OpenCAD
Read some of the control data from the EUAs
Add “Standard Samples” section to draft of Continuous Validation
Give feedback to OpenCell on their paper
Plot UK ratio to tests to new positive cases
Get ratio view on OurWorldInData, tweet it.
Respond to Michael Crone’s helpful message on testingmethods


Reply to some more tweets
Start reading ISO 13485
Read comparison between ISO 13485 and 9001
Open issue on OurWorldInData regarding seeming error in tests per positive cases
Open another issue for clarifying data range
Open another issue for allowing daily / moving average over data range
Contact Bev re Charlotte’s message to get NZ folks on testingsmethods call
Read ISO 13485 for an hour, check out patents filed conflicting with ISO.
Contact OpenCell re ISO 13485
Start reading CDC / PHE approved tests
Get distracted by seeing Human Specimen Control (HSC) (Extraction Control)
Diff the current list of FDA EUA tests with CCI list.


(life admin, cleaning house etc)
Read interesting vaccine research: https://science.sciencemag.org/content/early/2020/05/06/science.abc1932
Read a summary of ISO 13485:2016 from PECB
* change of use (plans may change)
* possible to act as validator company
* costs & time
Look up RepRap again (so good)
Read email from NHS England & NHS Improvement regarding talk tomorrow. Decide not to forward to Bomb.Bio team. Re-read Peter Stoilov’s comment. Tweeted it.
(life admin, take deliveries)
Some FreeCAD python tutorial
Tell Bomb.bio about broken links… want to send protocol “page” (the news page) to UK pathology lab manager.
Read about failure in RNA extraction (?) when blood in sample: https://bomb.bio/forums/topic/implementing-bomb-protocol-for-rna-extraction-in-colombia#post-9344
Reading paper on clinical sampling for SARS-CoV-2: https://jcm.asm.org/content/jcm/early/2020/04/03/JCM.00512-20.full.pdf
Also note: references to two papers now on GITC not being enough to inactivate ebola (one – thank you Catherine M, two – thank you Tim H)
Find some more papers on lysis buffers (add as comments to SARS-CoV-2 testing repo). Different standards for magnetic strength for RNA purification.
Make list of open-source scientists, medics, explorers, centred around Bomb.Bio for now. Post to Bomb.bio
Respond to messages from kind vascular surgeon.
Tweet Isle of Man use of Bomb.bio
Send to other lab managers.
Ask Bomb.bio permission to post message across multiple threads.


Start designing pipette
Watch tutorials on FreeCAD
(Sleep at 1 am)
NHS Improvements call, present Bomb.Bio technique in 4 minutes.
(1 hour stretch & meditate)
Thank Peter Stoilov et al. at West Virginia uni and inform them of their entry in the spreadsheet of Bomb.Bio users.
Translating Alejandro et al at AGROSAVIA’s independent validation of Bomb.Bio protocol and tweet it.


Slow progress understanding FreeCAD (lots of crashes)
Help OpenCell with write up (half an hour)
Draft request for help for open-source “reference implementation”. Take pictures of pipette.
Reading about CE marking beginning with classification of medical device, EU directives, find EU industrial strategy for increased domestic pharma (good).


Add Strategy section above
Sort out email address to get standards
Sort out domains
Get BSI standards
Read Crick protocol
Message Michael regarding Crick protocol


Read through https://rdml.org/ and spec.
Review 2009 Minimum Information for Publication of Quantitative Real-Time PCR Experiments (MIQE) guidelines https://academic.oup.com/clinchem/article/55/4/611/5631762
Review data on FIND.
Respond to Michael regarding Crick FPs
Mechanism of Background RT-qPCR amplification




Contacted authors of Korean CDC paper to request info on distribution of genders.
Read Basu et al. Abbott ID NOW paper.
Contacted authors regarding Cepheid Ct values stated in EUA.
Contacted Zach about his tweet: https://twitter.com/alchemytoday/status/1260937741572861953


Review OpenCell paper content on data management.
Read “Temporal dynamics in viral shedding and transmissibility of COVID-19”: https://www.nature.com/articles/s41591-020-0869-5#MOESM1
Read: https://rachomics.blog/2020/05/17/covid-testing-explainer-2-what-is-a-false-negative/
Reading, SARS-CoV-2 proteins & targets: https://www.nature.com/articles/s41586-020-2286-9_reference.pdf
Finish reading Crone’s paper: https://www.medrxiv.org/content/10.1101/2020.05.02.20088344v2.full.pdf
. Note: Unit (U) definition (ref: ThermoFisher TURBO DNase manual): One unit is the amount of enzyme required to completely degrade 1 μg DNA in 10 min at 37°C, and it is equivalent to 0.04 Kunitz units.
What video on Echo acoustic liquid handling: https://www.labcyte.com/resources/videos/echo-liquid-handling-labcyte
Respond to Michael regarding Crick FPs
Chat with Sam
Start spreadsheet on components of a validation paper


Watching video on Antha: https://vimeo.com/398988748
Reading paper on replicates: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3321166/
Start filling in “Test comparison data” spreadsheet on SARS-CoV-2 test comparison with first paper from Crone et al.


Read article on stats, intra & inter-run coefficient of variability
Add Crick paper to “Test comparison data” spreadsheet
Reviewing OpenCell paper & PR
Read up on calculating confidence intervals of test accuracy: https://www.hindawi.com/journals/cmmm/2016/7141050/
Try simulating data for TP FP FN TN
Translate v2 of Alejandro’s paper (they’ve already done it!…great)
Reading FDA guidelines on discrepancy resolution: https://www.fda.gov/media/71147/download


Promoting OpenCell’s preprint paper:
Reading up on power analysis for qualitative measure: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-018-0594-7


(Morning wiped out)
Looking at statistical methods for confidence intervals




More stats.
Talk with Dan


Reading Altman 2000 “Statistics with Confidence”
Talk with Joi & Karthik
IPScreener call / demo


NHS Improvements call.
Message David W.
Message Diane re IPScreener / patent IDs
Welcome Katie to slack channel
Post idea about “Cambrian” openness: corrosive & creative
Talk with Diane


Reply to IP Screener email.
PHE test shortcomings
(Finally) implement “recommended” Wilson’s method for confidence intervals.


Clear out tabs & spreadsheet of potential contributors
(meditate, lunch)
Watch video on OpenSource and patent risks: https://openuk.uk/past-events/
Read some of ISO 13485:2016
Roughly add form questions


(BLM news)
Call with Karthik & Joi re potential working group members
Call with CenterOfCI on testing projects progress
Call with Karthik & Alpesh re potential working group members
OCP call


(work out)
Research IEEE, standards process etc https://standards.ieee.org/develop/initiating-project/projstart.html


4allofus diagnostics conference
Jim’s heaters call
(work out)
BLM discussion
Cornell’s LAMP call
More BLM discussion
CCI call
More BLM discussion


Create a lens.org account
Query lens.org for Sandia patent ids






Find “not a vegetable”.


Some OCP & Sandia patent work.
Prep & CCI calls
OCP call


Do a lot on OCP and Sandia patents list.
IEEE societies


Read EP14 – Evaluation of Matrix Effects
Read BS EN ISO 15193:2009 (which supersedes BS EN 12286:1999)


Continue reading EP14 – Evaluation of Matrix Effects
OCP & Sandia work
Read BS ISO 20395:2019 — Biotechnology. Requirements for evaluating the performance of quantification methods for nucleic acid target sequences. qPCR and dPCR












Read ref 20 from Crone’s paper “Effect of large-scale testing platform in prevention and control of the COVID-19 pandemic: an empirical study with a novel numerical model”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s