Online symptom checkers get failing grades

"In an audit that is believed to be the first of its kind, Harvard Medical School researchers have tested 23 online “symptom checkers” — run by brand names such as the Mayo Clinic, the American Academy of Pediatrics and WebMD, as well as lesser-knowns such as Symptomate — and found that, though the programs varied widely in accuracy of diagnoses and triage advice, as a whole they were astonishingly inaccurate. Symptom checkers provided the correct diagnosis first in only 34 percent of cases, and within the first three diagnoses 51 percent of the time."

Harvard researchers tested 23 online ‘symptom checkers.’ Most got failing grades. Here’s how they stack up. - The Washington Post

Trext- interactive texting

Trext is a very interesting text (sms) based messaging service that has many potential uses in healthcare. Some examples that I can think up are: 

  • CHF - daily reminder for patients to check their body weight and text it back to the predefined phone number
  • Diabetes - frequent reminders to check and text back blood sugar levels. If the blood sugar is higher than a predefined level, then a healthcare provider could be sent a text message automatically using the same service
  • Medication reminders, especially for older people that are living alone. If they don't respond within a predefined time frame, a text message could be automatically sent either to a healthcare provider or a family member to check up on the person
  • Text based reminders for upcoming physician appointments, and the ability to reserve a place in line in healthcare facilities that see walk-in patients e.g. Urgent care centers 
  • Health campaigns and patient surveys using multiple questions with decision logic  

The kicker is that the data can be exported in an MS Excel spreadsheet to keep a comprehensive record a patient's health status. 

FAQ's - Hierarchical Condition Categories

(I wrote these FAQ's to share with the physician community in our ACO. Hopefully physicians practicing in other ACO's may also find these helpful.) 

What are HCC’s?

CMS groups certain related diagnoses under a single identifier, which are called Hierarchical Condition Categories (HCC's).

Related Diagnosis HCC Category Risk Adjusted Weight
Esophageal varices w bleed
Esophageal varices w/o bleed
Hepatic coma
Portal hypertension
HCC #25 1.779
Alcoholic Cirrhosis
Cirrhosis of Liver NOS
HCC #26 0.5292

What is the purpose of HCC’s?

HCC’s are used by Medicare and Medicare Advantage (MA) plans for risk adjustment of patients to determine their health/sickness status. Based on this risk adjustment, Medicare & MA plans determine how many dollars will be set aside to take care of patients in the following year. 

E.g. Coding for diseases that fall under HCC #25 tells the payors that the patient is sicker and will require more healthcare resources. The insurance companies will then set aside additional dollars to take care of these patients.

How does this benefit my practice?

Accurately documenting comorbidities allows insurers to use HCC risk adjustment model to set aside more dollars for individual patients in the following year. Under shared savings and risk based contracts, if more dollars are set aside to pay for healthcare, then there is a higher chance that there will be money left over at the end of year that could be distributed to the providers.

E.g. If we code accurately that the patient has “portal hypertension” (HCC #25), in addition to “alcoholic liver disease NOS” (HCC #26), the payors will set aside more dollars to take care of this patient in the next year. This increases the probability that there will shared savings.

How do I ensure appropriate HCC coding?

There are a couple of ways to ensure that we capture all the appropriate information for risk adjustment:

1. Be specific about the diagnosis in your Assessment & Plan and ensure accurate charge capture

E.g. Document the treatment plan for both “Alcoholic Cirrhosis” and “Portal Hypertension”. The treatment plan can be as simple as – “Stable, No medication changes.” Ensure that the ICD codes for both these diagnoses are captured on the bill that is sent to insurance companies. 

2. Document and bill every year that the patient still has the diagnoses, even if the treatment will not change. This ensures that the insurance companies will continue to factor the diagnoses when calculating payment for the next year


Year Diagnoses documented & billed Risk Adjustment Calculation for Next Year
2014 Alcoholic Liver Disease
Portal Hypertension
2015 Alcoholic Liver Disease 0.5292
2016 (Dollar allocation for 2016 may be lower than in 2015 as "portal hypertension" was not documented in 2015)

Based on the documentation/bill in 2014 (in the table above), the dollar allocation by insurance companies for 2015 is higher. However, the risk adjusted dollar allocation for 2016 may be lower as “portal hypertension” was not documented and billed in 2015.

Optimizing Clinical Decision Support

A recent paper in Journal of Hospital Medicine listed some practical and actionable lessons to help optimize clinical decision support (CDS). A brief summary is below: 

  • Defined metrics should be tracked over time to demonstrate progress and to make iterative improvements
  • Evaluate not only process measures but also outcome measures
  • CDS is a component of an educational program to guide and alter clinical behavior, and it must be deployed in conjunction with other educational tools
  • In teaching institutions, many orders that reflect questionable practices are entered by resident physicians, PA's, APRN's, and nurses who are least empowered to challenge requests from senior staff (therefore education is as important as CDS implementation)
  • Beware of alert fatigue. Ensure only that the necessary alerts are active
  • CDS must be maintained over time as clinical guidelines and clinician receptivity to each CDS evolves
  • Free text entered as part of structural data entry, or for overriding CDS offer significant insights on how to optimize CDS. These should be monitored systematically on an ongoing basis and used to improve CDS

CDS is a very effective tool to nudge and remind providers to practice evidence based medicine, however, it must be used judiciously to be effective.

Clinical Decision Support "against" Prostate Cancer Screening


Prostate cancer is the second most common type of cancer in men. The use of Prostate Specific Antigen (PSA) for cancer screening, however, has led to the over-diagnosis and over-treatment of prostate cancer. (Please see my earlier post  - Screening for Prostate Cancer.)

Implementing Clinical Decision Support (CDS) system has the potential to prevent unnecessary screening, reduce patient harm, and lower health care costs. 

Workflows and Trigger Events

To implement a successful CDS, a good understanding of the clinical workflow is crucial. A typical (simplified) office workflow for physicians is as follows:

  1. Patient arrives at office and registers
  2. RN or Medical Assistant takes vitals
  3. Patient is ushered in exam room
  4. PCP either reviews chart before entering exam room (to refresh memory) or reviews chart before making recommendations
  5. PCP makes recommendations to patient
  6. PCP places orders (medications or labs) in the EHR 
  7. PCP completes encounter record
                        PCP Office Workflow - CDS opportunities highlighted in green

                        PCP Office Workflow - CDS opportunities highlighted in green

Item 4 and 6 (those highlighted in green in the flowchart) in the above workflow are targets for CDS. 

Providers generally review the chart before talking to patients. Since, some providers may be unaware of the new prostate cancer screening recommendations, opening the patient's chart in the EHR should be a CDS trigger. This CDS can be in the form of a disruptive alert, or preferably a visual "non-disruptive" alert on the patient review (or snapshot) screen.

After the CDS is triggered, the EHR will run the risk stratification algorithm (outlined below in the article), and produce the desired output determined by the algorithm. 

After the CDS is triggered, the EHR will run the risk stratification algorithm (outlined below in the article), and produce the desired output determined by the algorithm. 

A second CDS opportunity when providers use Computerized Provider Order Entry (CPOE) functionality enter orders. The trigger would be when a provider places an order for a PSA test. The EHR can alert the provider about the new guidelines, and/or if a prior PSA test result exists.

After the CDS is triggered, the EHR will run the risk stratification algorithm (outlined below in the article), and produce the desired output determined by the algorithm.

After the CDS is triggered, the EHR will run the risk stratification algorithm (outlined below in the article), and produce the desired output determined by the algorithm.

An info-button next to an elevated PSA value can also be implemented. This may help providers interpret the PSA values in accordance to the guidelines.

Required Data for CDS

Several elements of data are required to create an effective CDS:

  • Patient Age
  • Problem list and Past Medical History:
    • To calculate 10 year survival probability using Charleson comorbidity index (or other survival scoring systems)
    • To look for prior diagnosis of prostate cancer, 
    • To estimate if patient is at high risk for prostate cancer (e.g. certain genetic conditions)
  • Family history - to determine if patient is high risk, especially if he has 1st degree relatives with prostate cancer
  • Last PSA result, and date test was performed  (if available)

Risk Stratification Algorithm

The risk stratification algorithm combines the Prostate Cancer Screening recommendations from the major clinical societies including USPSTF, AUA, ACS and ACP. This is a basic algorithm, which will need testing before implementation.

Measuring Effectiveness of CDS

It is essential to measure the effectiveness of any new CDS pathway to determine that it is producing the anticipated results. A simple way to test the PSA screening CDS is by measuring the number of PSA tests ordered in patients over 50 years old. One important exclusion criteria (i.e. do not count PSA tests ordered on patients) is patients with prostate cancer. PSA is sometimes used to follow treatment and recurrence of prostate cancer. Furthermore, CDS override reasons should be monitored, and if justified, the reason should be built into the exclusion criteria.


Prostate cancer screening using PSA test carries more risk than benefit. Implementing a well designed clinical decision support system will reduce unnecessary screening, and promote shared decision making based on risk factors, thereby reducing patient harm.