[rgejman]

This website almost certainly violates FDA rules on human subject experimentation because it collects data on the effect of a drug on human subjects. I would stop working on this project until you have advisors (and lawyers) who have experience with regulatory oversight from the FDA. At the bare minimum you are going to need (a) procedures to gain informed consent from trial enrollees and (b) an Institutional Review Board (IRB) to sign off on your trial. Depending on the drug, you might also need an FDA waiver to conduct the trial.

The PhD student (Rob Wagner) listed in the about section should have had at least one course on the responsible conduct of research. This course is required for most (all?) scientists working under a training or research grant from the NIH. Design and implementation of clinical trials--along with the ethical and legal issues involves--is standard fare for these courses for precisely this reason!

[chrisrxth]

Thanks for the thoughts. We've spoken with a lawyer who specializes in FDA issues. We will have informed consent waivers for participants.

Rob does have formal training in scientific ethical issues and is more than competent on the subject.

Also, L-Theanine is something that you can purchase at the grocery store as a food supplement and it has an excellent safety record and no known toxic dosage.

If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.

[rgejman]

I don't think your platform is exempt from the "common rule" requirement to obtained oversight from an IRB.

The fact that L-theanine is obtainable over the counter is not material. If you study the effects of aspirin on cancer outcomes, you must still obtain consent and IRB review.

A supplements distributor that records whether supplements work is still engaging in human subjects research.

[Defenestresque]

>If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.

From your website:

>we want to make this clear: we are not a supplement company. We are a personal experimentation platform.

That being said, I actually like your idea. You just need to be more careful about choosing a message (and staying on message).

[OopsCriticality]

> If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.

You are then, by definition, conducting human research.

[ska]

Cool idea, but as many have pointed out there are some potential issues with the execution.

I'd suggest you get an additional formal legal opinion on applicability of HIPPA as well, it would be financial painful (up to something like $250k/record if I recall correctly) for you to get caught on the wrong side of that one.

You should also (already) have a proper formal ERB.

[mattwinslow]

The other comments are on point. From my perspective as someone in the digital health / clinical research industry, the site (as is) raises a bunch of red flags:

- There's no indication this study has received IRB (Institutional Review Board) approval, which is a baseline. Here's a handy reference when asking the question, "Does my research need IRB approval?": http://www.irb.umn.edu/research.html

Based on several factors, the study/experiment qualifies as research (it meets the requirements for being a systematic investigation, contributing generalizable knowledge, and involving human subjects). IRB review/approval can be fairly straightforward, but does cost a non-trivial amount of money.

Chris, if you need to work with an IRB, I recommend getting in touch with Copernicus Group (http://www.cgirb.com/). I've worked on 20+ projects where they've been the IRB and they've always been responsive and pretty open to innovative ideas.

Echoing rgejman, if you don't have IRB approval, stop everything. Now.

- There's no mention on the site of participants completing an informed consent, which should really be done prior to collecting their money for the kit.

- Has any thought been given to how the team will handle adverse event reporting? This deserves serious consideration and is sometimes overlooked (there are a slew of FDA guidelines/regulations around this). Even for something that seems benign or where "nothing could ever go wrong" - you'd be surprised.

- The site doesn't instill a ton of confidence that data security is a high priority. Using SquareSpace to host the public facing informational portion of the site isn't necessarily a problem, but it leads me to wonder how the team is handling surveys and how the survey data will be stored. It seems like participants will likely provide personally identifiable information (PII) and personal health information (PHI) within surveys, and that's when things start to get a bit more interesting.

It's certainly possible to use an off the shelf survey tool (like SurveyGizmo), but there are a few more hoops you'll have to jump through. Take a look at the following link: https://www.surveygizmo.com/survey-blog/online-survey-hipaa-...

- There doesn't seem to be a privacy policy. The About Us page mentions, "We will have a complete policy online before we start taking orders for the first experiment." The SquareSpace checkout appears to be up and running, but I'm unable to find the privacy policy.

There's a more or less standard process for designing, developing, and getting a study out the door (highly simplified, and skewed heavily towards direct to patient studies): 1. Develop your protocol. 2. Get your protocol approved by an IRB 3. Develop/configure the tech to implement the mechanics outlined in the protocol (this can be done in parallel to IRB review and/or protocol development) 4. Recruit patients 5. Collect informed consent, enroll patients 6. Collect data 7. Everything else...

One of the tricks here is that all of the content a patient might see has to be IRB approved.

Want to run ads soliciting your study? They have to be IRB approved.

Want to optimize your ads? Make sure your content is submitted modularly for review (ie, list all your headlines, body content, and images separately - and throw the kitchen sink at any variations you can think of to save yourself another round of review).

Want patients to pay for their kit via a SquareSpace checkout? Probably requires IRB approval, and likely needs to occur after a participant has completed an informed consent doc.

Need to send password reset emails to patients in the event they can't remember their login to access the study survey tool? Has to be IRB approved.

If you wanted to use efficacy tracking as a marketing tool for your supplement business, you might have a bit more leeway if you were to reposition what you're offering. You could (and there's a good chance I'm wrong as this is off the top of my head) simply offer your customers a tool to track the effects of their supplements. This would also help you with reorders by providing an adherence mechanism.

Unfortunately, you'd likely have to scrub using the Double Blinded site, referring to what you're offering as an experiment, or mentioning that the results will be aggregated and published. Data protection issues would still be a serious consideration though.

All of that said, the concept of citizen science and open/transparent studies is very cool. It's just a (minor) hassle to dot all the i's and cross all the t's.

Also, brilliant idea to offer customers a way to track efficacy. I've been toying with similar ideas, so it's cool to see someone else implementing something.

[volkhavaar]

This is an incredibly informative and useful post. Thank you.

[mattwinslow]

Thanks! Digital health can be incredibly fun and rewarding, but there are plenty of opportunities to run head first into walls you may not have realized even existed.

Aside from the ethics considerations, privacy regulations (and the desire to tread conservatively) can lead to some odd implementation decisions...especially if you're coming into the industry from a general or consumer oriented tech background.

I once worked on a medication adherence project for patients in Italy where patients needed to complete bi-weekly assessments and generally track their medication adherence. We set up a secure portal/website for patients, but were prohibited from emailing them reminders when their next assessment was due.

The sponsor (aka pharma co) was extremely conservative re: patient privacy laws and more or less left it at, "well, the patient will just have to remember to return to the website, log in, and complete their survey at the appointed time." After a couple of months of sub-optimal survey completion rates, they re-evaluated their position and let us send out survey reminder emails.....which is a baseline thing to do when you need to retain participants over the life of a study.

Oddly enough, this was for a commercial project (medication was available on the market, physicians were referring their patients to this website as a value added sort of service) and the commercial side is a bit more wild west than phase 2/3 research. Well, as wild west as the pharma/clinical research industry gets :)

There are parallels between customer acquisition/retention in SaaS and patient recruitment/retention in clinical trials that are pretty interesting...but I think most folks on the Saas side would think the healthcare folks are trying to solve problems with one (if not two) hands tied behind their backs - and in some cases, wouldn't be wrong in thinking so.

[reitanqild]

> If it makes you feel better, we are a supplement distributor that happens to help you record whether or not the supplements actually work.

I liked that statement a lot, at least I think it is excellent marketing!

[rgejman]

As an aside: it's a cool idea and worth trying--but you are going to need to make sure that you are compliant with this country's laws regarding the responsible conduct of human subjects research!