In our initial post reviewing the recent article in Evidence Based Medicine (BMJ Group), we provided an overview of the problem as explained by the authors of this paper. In this post, we summarize their recommended solution for establishing quality control, safety, and clinician confidence in the utilization of medical apps.
For reference, here is the article again: “Medical apps for smartphones: lack of evidence undermines quality and safety” that was published in Evidence Based Medicine in August 2012 [ePub ahead of print - not open-access]:
PUBMed Link: http://www.ncbi.nlm.nih.gov/pubmed/22923708
Article Review Part II: The Solution
We discussed some of the concerns that were raised regarding authenticity (the credentials of the author, reliability of the information, lack of references, security, etc.) I also wanted to reiterate some of what was mentioned in the comments and discussion following this first post. One of the points being that there is (and has been) a proliferation of textbooks and medical handbooks available for sale in bookstores and on the Internet. This is important to remember, because although the author’s name is always transparent, there is still no guarantee as to the nature of the content, how well it has been referenced, or whether it represents best-evidence content for its subject. There is still significant controversy over various aspects of medicine, and we certainly have not arrived at a place in medicine where consensus on everything is universal. Additionally I mentioned the list of online calculators and resources that many clinicians access, which currently have very little if any regulation or validation process. As we jump forward to make a case for the need to validate/regulate apps, let us remember the current state of affairs.
The authors of the EBM paper discuss possible solutions around various issues, but overall seem to make the case for a multidisciplinary approach, involving various stakeholders, to include……..
American Food and Drug Administration (FDA): We’ve written in the past about the draft guidance for regulating apps that the FDA has published (you can read more here), and this has been a target of much press and debate lately so I won’t revisit this in detail. While these authors agree that some regulation is important, they reiterate what I think many of us feel as well, that “government health authorities should not over-regulate medical apps so as to retain their open nature.” And, what we believe to some extent as well, is that “the regulation process should be managed primarily by the healthcare community itself.”
Where I think the authors over-simplify the problem is in this:
“However it would be beneficial for government health authorities to provide official certification marks guaranteeing the quality of apps so that physicians can make an informed choice as to whether an app has evidence-based reliability”
I believe certification will be a challenging process, as in many cases new versions of the same app come out on almost a weekly basis. There are other issues such as determining which apps require certification and what features of the app need certification. I think those closest to making this a reality are at Happtique (a non-federal entity), who is currently rolling out such a process this fall. It will be interesting to see how they’ve managed to progress through some of these issues, and I think we’ll be able to learn a lot from the processes they will have implemented.
However, where I think the statement from the authors a bit naïve, is in the thought that certifying apps will allow physician to make an informed choice about “whether an app has evidence-based reliability”. We’re learning through the establishment of our medical app peer-review process that this is often challenging to implement. How many references do you need for an anatomy app for example? (Although, one of our reviewers who is the head anatomy instructor at the University of Colorado School of Medicine, has found a couple different variances in terminology in some of the apps he has reviewed). What about the variety of clinical practice guidelines out there, and the fact that there is poor adherence by clinicians to many guidelines (for example in the management of low back pain where diagnostic imaging is over-utilized, narcotics are over-prescribed, and providers continue to ignore the evidence in this area)? Guidelines may vary slightly from different countries and new evidence comes out constantly. And maybe even most importantly, there is often disagreement within the medical community regarding what is best-evidence practice. Ensuring that information in apps are always representative of the most current evidence, even if there were universal consensus on all recommendations, would be an enormous and daunting challenge.
Publishers: Another recommendation the authors of this study make is to get medical publishers involved. However, publishing companies are often counting on the “subject-matter-expert” author they’ve signed a contract with to provide them the information to publish. That’s historically how the relationship works, and if a given author is unable to validate the nature of their content, or the publisher in some way has to verify it’s usefulness based on their own clinical resources, you could argue that the author may not even be needed in the first place. It is most certainly a business-relationship, just as it has always been in the traditional textbook publishing model. Many traditional book publishers have already entered the app scene and are delivering much of their content either directly (Elsevier) or indirectly with app development partnerships (Unbound Medicine, Medhand International, Skyscape, etc). In addition, and most important to keep in mind, is that having publishers regulate their own content would result in a significant conflict of interest.
Hospitals & Healthcare Systems: The other solution they present is to have hospitals and healthcare systems involved in the curation process, ultimately providing their own “app stores” where clinicians in their network would in theory only be able to access apps that have been approved by the hospital. In theory I think this may sound like a valid solution, but I see several loopholes in this proposed solution from a pragmatic perspective. First of all, each hospital will still have the need for a curation process that will need to address all of the issues that have been discussed so far. The cost as related to time, personnel, and resources required to maintain an established and ongoing approval system will not be trivial. But, the other challenge will be that of the individual provider. A clinician is taught to be an independent thinker and problem-solver. The skill of appraising the literature and being open to new developments in medicine is a welcome one. Whereas it is nice to have resources to help with this process, I’m not sure that every clinician wants complete autonomy taken away from them. I would anticipate that if a hospital does not do a good job of curating the apps in their store, then they will begin to lose credibility with clinicians. For example, if I hear about a new app that sounds interesting to me, but then I don’t see in my hospital’s approved app store, probably out of curiosity I would be inclined to go and find it elsewhere. I think I can trust my own clinical intuition and appraisal skills to determine if there is some merit or potential use for the app. Will there be a “grass is greener on the other side” sentiment about this if I’m tied to only using apps within my app store? Will clinicians wonder what “other apps” are in the regular app store that are not found in their hospital’s approved app store? While I’m not saying there is no value with this solution, I do think that the implementation process will have to be extremely sound for there to be success with this model.
Finally the authors in this paper show a figure representing the various stakeholders that should be involved in the regulation process of medical apps:
- Medical Device Companies
- Hospitals and Healthcare Institutions
- Government (Policymakers and Regulators)
- Physician accrediting and Organizing Bodies
- Medical Publishers
- Healthcare Professionals and Executives
- Pharmaceutical Industry
Whatever the best solution is, it will likely require input from various stakeholders
In my opinion, one crucial component is missing from their list: the consumer (in our case the clinician, but also the patient when health apps are concerned) – should they not have a voice in this as well?
Just like many other things, a balance will be important. Too much power in the hands of one entity can cause problems such as stifling innovation (FDA) or introduce bias and/or conflicts of interest (Pharmaceutical and Medical Publishers).
Ultimately, while it will be nice to have some sort of filter and some guidance for making decisions, what it comes down to is that clinicians need to continue to learn and master the art of being wise consumers of medical literature and information. There will be no valid answer for this problem that completely removes thought, clinical reasoning, and experience from the equation (and I don’t think there ever should be).