Further Validation of an Opioid Risk Assessment Tool: The Brief Risk Questionnaire
- 1. Behavioral Medicine Institute, University of Tennessee Knoxville, USA
Abstract
Opioid risk assessment and risk stratification has become a standard of care when prescribing opioid medications for chronic pain conditions. Research to date has shown that different risk assessment tools yield different accuracies in predicting future medication aberrant behavior. This study offers further validation data on a new opioid risk assessment tool, the Brief Risk Questionnaire (BRQ). The BRQ was compared to the Brief Risk Interview (BRI), the Opioid Risk Tool (ORT), and the Pain medication Questionnaire (PMQ) in their ability to predict medication aberrant behavior at six-month follow-up. Two hundred ninety-nine (299) patients were assessed. One-hundred forty-two patients were later treated with opioid medications and the presence or absence of medication aberrant behavior was recorded at six month follow-up. Results found that the BRQ was able to predict future medication aberrant behavior as well as other risk measures and appear as good an overall predictive tool as other commonly used measures. The BRQ has less specificity when compared to other patient-completed risk assessment tools and the implications of this are discussed. This study indicates that the BRQ could be a useful tool for clinicians in conducting opioid risk assessment.
Citation
Jones T, MSchmidt T, Moore T (2015) Further Validation of an Opioid Risk Assessment Tool: The Brief Risk Questionnaire. Ann Psychiatry Ment Health 3(3): 1032.
Keywords
• Opioid risk assessment
• Medication aberrant behavior
• Behavioral risk
INTRODUCTION
Risk stratification of patients with chronic pain, before opioid medications are initiated, is now a national standard of care. Use of a validated screening tool is generally recommended as a way of accomplishing this assessment [1]. Various risk assessment screening instruments have been offered for clinical use, including the Opioid Risk Tool (ORT) [2], the Pain Medication Questionnaire (PMQ) [3], the Diagnosis, Intractability, Risk, Efficacy score (DIRE) [4], the Screener and Opioid Assessment for Patients with Pain (SOAPP) [5] and its revision (SOAPP-R),[6] and the Brief Risk Interview (BRI) [7]. These tools vary in how they are conducted (some with interview, some by staff rating and some as a written patient questionnaire), but all have been offered for clinicians’ use in conducting risk stratification.
The issue is an important one as Drug Enforcement Administration (DEA) regulations call for prescribers to “exercise a much greater degree of oversight to prevent abuse and diversion in the case of a known or suspected addict” than in the case of a patient for whom there are no indicators of drug abuse [8]. Very likely the DEA also believes increased oversight is needed for anyone suspected for abusing their medication as well (not just “addicts” per se). Opioid risk assessment then is a recommended and important tool that clinicians are expected to use according to federal regulators. In addition, some insurers are now asking clinicians to use risk assessment results to determine how often to monitor patients with urine drug tests (higher risk patients should be tested more frequently) [9]. Thus, risk assessment and risk stratification are important processes that any clinician should engage in when prescribing opioids, and the risk assessment tool or process that is chosen by the clinician has important ramifications for various aspects of clinical practice.
However, comparative studies of various risk assessment tools have found significant differences in the tools’ ability to predict medication aberrant behavior. Moore, Jones, Browder, Daffron and Passik (2009), found that a clinical interview and the SOAPP performed significantly better than the ORT and the DIRE in identifying which patients would engage in medication aberrant behavior [10]. A subsequent article in 2012 offered two separate studies that compared several risk assessment tools [11]. The first study in that article revealed that a clinical interview was a more sensitive method of risk assessment when compared to the ORT, PMQ and SOAPP-R. The second study assessed both sensitivity (correctly identifying patients who later show medication aberrant behaviors) as well as specificity (correctly identifying patients who later do not show medication aberrant behaviors) of four risk assessment tools: a clinical interview, ORT, PMQ and SOAPP-R. This study found that the clinical interview outperformed other risk assessment tools in sensitivity as well as overall predictive accuracy. Another comparative study in 2013 found that a structured clinical interview and rating system (the Brief Risk Interview: BRI) outperformed the ORT and SOAPP-R in predicting future medication aberrant behavior [7]. A recent study has replicated this finding that the BRI outperforms the ORT and SOAPP-R in predicting future medication aberrant behavior [12].
The overall results of these studies find that there are significant differences between various opioid risk assessment tools and that a specific clinical interview has shown the best overall accuracy in predicting future medication aberrant behavior. However, while the brief clinical interview may have shown the best predictive results to date, many pain clinicians may feel they do not have sufficient time or staff to conduct an interview. A written patient questionnaire requires less staff time and training and is more easily adopted into a pain practice.
Towards this end, a new patient-completed questionnaire was developed: the Brief Risk Questionnaire (BRQ). The BRQ is conceptually based on the Brief Risk Interview (BRI) as it asks for information about 12 areas of inquiry thought to contribute to patient’s risk of misuse of opioids. A recently published validation study found that the BRQ is roughly comparable to the BRI in predictive accuracy and was superior in predictive accuracy to the ORT and to the SOAPP-R [13]. The current study was designed to offer further validation data on the BRQ and serves as a replication of the initial validation study. In this study the predictive accuracy of the BRQ was compared in a new patient sample to the BRI as well as to two other opioid risk assessment tools, the Opioid Risk Tool (ORT) and the Pain Medication Questionnaire (PMQ). The study hypothesis was that the newly developed BRQ would show adequate predictive accuracy as compared to two other patient completed questionnaires.
METHODS
The Brief Risk Questionnaire (BRQ) © (Jones, Lookatch & Moore, 2015) is a 12-item questionnaire based on the BRI, (Figure 1), an interview schedule that has shown good predictive results [12,13].The BRI and the BRQ were conceptually designed to predict opioid misuse, abuse, addiction and diversion of opioids – a broader array of behaviors than only predicting opioid addiction. The BRQ asks a single question about the areas of past discharge from treatment, overtaking of medication, street medication use, depression and anxiety, presence of Bipolar Disorder or Attention Deficit Hyperactivity Disorder, medication security, personal history of substance abuse, family history of substance abuse, history of legal issues and intellectual and literacy issues. The BRI also addresses the issues of patients “pushing” for certain medications and about patient honesty. As these are behavioral issues, they could not be effectively pulled into the BRQ, making the BRQ slightly different in content areas addressed from the BRI. The BRQ was developed to be succinct and easily understandable while inquiring about multiple important content areas related to medication aberrant behavior.
The BRQ is scored by giving different weight or points to the respondent’s answers. For example, on BRQ item one “Have you ever been discharged from a practice,” “No” is given 0 points while “Yes” is given 2 points. “How is your reading ability” is scored with 1 point for “Can’t read” and the other two answers are given 0 points. Each answer on the BRQ has a point value. The item scores are summed for a total BRQ score. BRQ total scores range from 0 to a maximum of 24.
After finalization of the BRQ format and content, the study was approved by the IRB of the University of Tennessee-Knoxville. The BRQ was administered within a psychology assessment packet and administered to 299 consecutive patients referred to a psychology practice working in association with a medical pain practice. Comprehensive treatment for chronic pain is offered at the practice and can include continuous opioid therapy (COT). Patients referred to the psychology practice were being considered for opioids as a part of their treatment plan for a chronic pain condition at the medical practice, and an opioid risk assessment was requested in the referral. Patients being treated with interventional treatments only and not being considered for COT were not given the assessment packet or referred for opioid risk assessment. If patients were being considered for COT, patients were given an assessment packet to complete and to bring to the psychology evaluation session. If patients forgot or did not complete the packet ahead of time, they were asked to complete it at the time of the psychology appointment.
The packet contained the Brief Risk Questionnaire (BRQ), the Opioid Risk Tool (ORT) and the Pain Medication Questionnaire (PMQ) as well as these other assessment tools: the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-item (GAD-7), the Pain Beliefs and Perceptions Inventory (PBPI), the Pain Catastrophizing Scale (PCS) and the Tampa Scale of Kinesiophobia (TSK-13) - the latter five tools being used to help plan other aspects of the patient’s treatment. The assessment packet was the same for every patient and the order of administration of all assessment tools was the same in every case. The completed packet was obtained from the patient and set aside by the clinician without review and a clinical interview conducted. The 45-minute interview by one of the two pain psychologists included questions for the opioid risk assessment portion of the evaluation (the BRI) and also covered other topics which addressed patient psychosocial pain treatment needs, such as the need for psychotropic medication, psychiatry referrals, and/or psychotherapy sessions to help with pain coping issues. An overall opioid risk evaluation rating was obtained based on the BRI results and was given to the medical staff for their use as they considered the use of opioids for the patient’s chronic pain condition.
Patient medical records at the pain practice were reviewed by study researchers six months after the psychological evaluation process. Information was gathered from the record, including patient demographics, general medications and treatments provided to the patient, the disposition of the case at the six-month follow-up and the presence or absence of medication aberrant behavior during the six-month follow-up period. A complete list of medication aberrant behaviors as defined for this study is given in Figure 2. However, these behaviors were then combined into nine items based on the similarity of the behaviors and this list is indicated in Table 1. Medication aberrant behavior was defined as documentation of a patient failing a urine drug test (UDT) (positive for non-prescribed opioids, negative for prescribed opioids, or positive for illicit drugs or alcohol), failing a pill count, obtaining opioids from another prescriber in violation of the treatment agreement (through checking the state prescription monitoring program or by some other information source), or a patient report of behavior that violated the treatment agreement. The latter could include a report of loss or theft of opioid medication, a report of overtaking medication or a report of giving medication to someone else. Troublesome interpersonal behaviors such as cursing, yelling at or threatening staff, though rare, were also included and counted as a medication aberrant behavior. Medication aberrant behavior sometimes was found in the initial UDT of the patients when a UDT found an unexpected result (i.e., positive for unreported or illicit substances or negative for prescribed medications). The number of times a medication aberrant behavior occurred over the follow-up period was recorded for each patient.
The presence or absence of medication aberrant behavior was determined in the following manner. All patients being prescribed opioids are seen monthly at the clinic per state law. All patient encounters are documented in the practice’s medical record (an Electronic Medical Record or EMR). The pain practice’s EMR notes are highly structured and have a section for documenting the presence or absence of medication aberrant behavior. If a patient ever shows medication aberrant behavior during treatment, it is recorded in that section of the note and retained from visit to visit with specific data about its occurrence. A pill count of all prescribed opioid medications is done at each patient visit, per state law in Tennessee. If a patient is short on the pill count by more than a day’s worth of medication, this is considered by clinical staff as failing a pill count and this is noted in the aberrant behavior section of the note. Having more pills than expected is not considered a failure but the medication dose or treatment plan is adjusted accordingly. The practice has a protocol of administering unannounced drug screens (either urine drug screens or sometimes Oral Fluid Tests; OFT). If there is an unexpected finding, the screen is sent for laboratory confirmation. A UDT or OFT result is counted as medication aberrant behavior after there is investigation by clinical staff as to whether the findings are truly inconsistent with what was expected. The drug screen from the initial patient encounter (the new patient evaluation) is always sent for confirmation, per state regulatory guidelines. The frequency of UDT’s done based on the risk assessment result (using the BRI results) with higher risk patients receiving more frequent testing. Low risk patients are tested twice a year (per state law) while high risk patients are tested at every visit, and the other levels are titrated in between these extremes. Very High risk patients are not offered opioids as a part of their treatment. The state prescription monitoring database – the Tennessee Controlled Substances Monitoring Database (CSMD) – is checked at the initial patient encounter and at patient visits at a frequency commensurate with the UDT testing. The results of the CSMD check, when done, are noted at the patient encounter while drug screen and confirmation results are noted at the next patient encounter note (when the results have come back from the laboratory).
Patient behavior and patient report are also noted at the patient encounter. If the patient reports overtaking medication, reports obtaining opioids from another provider and this was not approved by this practice, or reports some other violation of the medication agreement, then this is noted in the patient record under the aberrant behavior section. Notable and out of the ordinary behavior at the encounter, such as cursing staff or becoming belligerent with the staff, is recorded in the note under the aberrant behavior section as well. In addition, when medication aberrant behavior is demented by a clinical staff member (a nurse practitioner), that staff member completes a “Discharge Review Form.” This form describes the medication aberrant behavior, offers some patient history, and makes a recommendation about how the treatment plan might be altered in light of the medication aberrant behavior. Options considered can include increased monitoring, increasing or decreasing or changing the opioid medication, referral to psychology or discontinuation of the opioid medication altogether. On this form two other clinical staff members (nurse practitioners) weigh in on changing the treatment plan and then these recommendations are routed electronically to the attending physician who makes the final decision about how the treatment plan should be changed and if opioids should be discontinued.
For this study one of the authors (TJ) reviewed the patient record six months after the risk assessment was done. If treatment did not last six months then the note from the last patient encounter was reviewed for this study. Specifically, the section of the last EMR note was checked for documentation of medication aberrant behavior over the course of treatment. As noted above, the notes are cumulative such that if a patient engaged in medication aberrant behavior in the first month, the record retained this information and it was still documented in the six month’s (or last) note. Since a Discharge Review Form was completed once a medication aberrant behavior was identified, then the presence of a Discharge Review Form in a patient’s record helped to indicate for the present study that there was medication aberrant behavior during that patient’s treatment. If medication aberrant behavior was documented in the EMR, then this was recorded on a treatment summary sheet used in this study and later entered into an Excel spreadsheet for analysis. The staff who reviewed the patient record was blind to the results of the three written patient-completed risk assessment questionnaires but was not blind to the BRI risk assessment results. That staff had no input into the decision to discharge the patient from care and had no role in identifying medication aberrant behavior during the course of treatment.
Table 1: Frequency of Medication Aberrant Behaviors.
n | % | |
Failed UDT for alcohol or illicit drugs | 6 | 9% |
Failed UDT for non-prescribed opioids | 19 | 28% |
Failed UDT negative for prescribed opioids | 2 | 3% |
Non-UDT data about use of alcohol or illicit drugs (from other providers, PMP, or patient report) | 0 | 0% |
Non-UDT data about use of opioids from other sources (from other providers, PMP, or patient report) | 1 | 1% |
Short pill count, out, lost or stolen medication | 34 | 49% |
Inappropriate behavior (not honest, threatening, excessively angry) | 1 | 1% |
Non adherence (did not follow treatment plan, multiple no shows) | 6 | 9% |
Total medication aberrant behaviors | 69 |
DATA ANALYSIS
Data were analyzed in Excel and in SPSS 22.0. For the purposes of study analysis the BRQ, ORT, PMQ total scores and BRI ratings (six ratings ranging from Low to Very High) were categorized into dichotomous categories of “Low Risk” and “High Risk.” The BRQ total scores of 0-2 were categorized as “Low Risk” while scores of 3 or greater were categorized as “High Risk,” folding Medium risk scores into the High category for purposes of analysis. Scoring values were used per the BRQ’s original validation study [14]. ORT total scores of 0-3 were categorized as “Low Risk” and scores greater than 4 were categorized as “High Risk,” again folding Medium risk scores into the High category for purposes of analysis. PMQ ratings of “OK” (total score < 25) were classified as “Low Risk” while “May have a problem” (≥ 25) and “Monitor closely” (≥ 30) were classified as being in the “High Risk” category.
The Brief Risk Interview, which was embedded within a 45-minute psychology interview, yields a risk rating of one of six levels (Low, Low Medium, Medium, Medium High, High and Very High). To facilitate data analysis the interview ratings were collapsed into “Low Risk” and “High Risk” by combining the Low and Low Medium categories into “Low Risk” while combining all other categories (Medium, Medium High, High and Very High) into the “High Risk” category. In the initial analysis any items left blank on any risk tool were coded as “0” and counted as part of the total score.
RESULTS
A total of 299 patients were evaluated initially with the four risk assessment tools. Of the 299 patients on whom evaluation data were gathered, 157 (53%) were not offered opioids or were prescribed opioids but did not return after the first prescription. A total of 142 patients(47% of the initial 299) were prescribed opioid medication for at least one month during the follow-up period and had at least one return visit to the clinic. A patient who was prescribed opioids but never returned for a follow-up visit was not included in the follow-up data of the 142 as there was no opportunity to note whether any medication aberrant behavior occurred or if the patient even ever took the medication. It was this group of 142 patients who were prescribed opioids at least once and had at least one follow-up visit that is the centra subject of the study analyses below, as these were the patients that had at least some opportunity to show medication aberrant behavior or not.
The 142 patients were predominately Caucasian (94%), roughly the same percentage as the local geographic population. The study population was 58% female and 42% male. The mean age of the 142 patients was 54years with a range of 20 years old to 89 years old. The primary pain complaint was low back pain (54%) followed by pain in a specific joint (e.g., knee, shoulder) (16%), neck pain (10%), arm or leg pain (9%) and abdominal or pelvic pain (3%). Of the 142 who were prescribed opioids and followed for at least one month, 102 (72%) were prescribed a short-acting opioid medication and 77 (54 %) were prescribed a long-acting opioid medication (some were prescribed both so the totals do not add to 100%).
Follow-up data on the presence or absence of medication aberrant behavior revealed that medication aberrant behaviors were observed in 48 (34%) of the patients prescribed opioids in the study. The frequency of medication aberrant behaviors is noted in Table 1. Not having the correct amount of medication either due to being short, out, lost or stolen was the most common medication aberrant behavior type (49%). Having a UDT positive for non-prescribed opioids was also fairly common (28%) Failing a UDT for illicit drugs or alcohol (9%) and non adherence to the treatment plan (9%) round out the top four categories of medication aberrant behavior. A total of 22 patients (46% of those that exhibited a medication aberrant behavior) exhibited more than one medication aberrant behavior over the six-month follow-up period. As to case disposition, 68% of the 142 patients were still in treatment at the end of the six-month follow-up period, 13% had dropped out of care and 16% were discharged for medication aberrant behavior.
The overall predictions of all four risk assessment measures studied from this analysis are presented in Table 2. Sensitivity is the accuracy of a measure to identify those patients who later engage in medication aberrant behavior at some point in the follow-up period. The most sensitive of the four measures studied here was the BRQ (73%). The BRI followed at 69% while the other two measures were much lower in sensitivity at 35% (PMQ) and 25% (ORT). A measure’s ability to identify patients who do not engage in medication aberrant behavior during the follow-up period is specificity. The PMQ had the highest specificity at 86% followed by the ORT (83%), the BRI (45%) and the BRQ (40%). Table 2 also shows an overall predictive value, the Area under the Curve (AUC) calculated from a combination of sensitivity and specificity values. AUC is an index of the performance of a measure that predicts a binary outcome (“hit or miss”). An AUC of .5 indicates random prediction and 1.0 is a perfect prediction. The PMQ had the highest AUC (.608) followed by the BRQ and BRI (.567) and the ORT (.540).
Missing data was present in one or more items of the PMQ and/ or BRQ of 24 patients (17% of this sample). Because the clinician made a point of not reviewing the written risk assessment tools when they were turned in (so as to be blind to the results) a number of items went unmarked and the patient was not asked later to complete them. As the BRI is an interview and the ORT counts unmarked items as “No” these two tools did not have any missing data. It could be that counting blank items as “0” (the way the above analysis was carried) causes significant errors in prediction, though previous studies on the PMQ did not state how patients with missing data were handled (if they were excluded from analysis or not) [3,14]. In Table 3 the predictive results of all tools were calculated by omitting any patients who had any missing responses. These data find that the relative predictive abilities of the tools are essentially unchanged if patients with missing data are excluded from analysis. These data indicate that clinicians can count missing items as “0” and still use these risk assessment results rather than discarding the entire assessment tool. This finding may support the use of risk assessment tools in a real world setting in which occasional missing data can be expected.
The use of a three-level risk assessment BRQ score (Low, Medium and High) was assessed. Cutoff scores of 0-2 (Low), 2-9 (Medium) and 9 or above (High) were analyzed, per the previous validation study of the BRQ [13]. Results of this three-level risk rating as displayed in Table 4. These data indicate that a score of 8 or below is associated with a 90% chance that the patient will not later engage in medication aberrant behavior. Use of a Medium risk category allows clinicians some flexibility in their assessment and use of a risk score.
Table 2: Initial Risk Rating and Predictive Results of Four Risk Tools by Presence of Medication Aberrant Behavior.
Medication Aberrant Behavior Absent | Medication Aberrant Behavior Present | Total | AUC | |
BRQ Low Risk | 38 (40%)* | 13 (27%) | 51 | .567 |
BRQ High Risk | 56 (60%) | 35 (73%) | 91 | |
ORT Low Risk | 78 (83%) | 36 (75%) | 114 | .540 |
ORT High Risk | 16 (17%) | 12 (25%) | 28 | |
PMQ Low Risk | 81 (86%) | 31 (65%) | 112 | .608 |
PMQ High Risk | 13 (14%) | 17 (35%) | 30 | |
BRI Low Risk | 42 (45%) | 15 (31%) | 57 | .567 |
BRI High Risk | 52 (55%) | 33 (69%) | 85 | |
Totals | 94 | 48 | 142 |
*Column percentages
CONCLUSIONS AND DISCUSSION
The results of this study found that the total score of the 12- item BRQ was comparable to the ORT and the PMQ (two other patient completed risk assessment measures) in overall predictive accuracy. These results support the findings of an earlier study that the BRQ is as good a predictor of MAB as any currently available opioid risk assessment measure [13]. The BRQ tends to be sensitive and identifies many patients who later show MAB. However, it appears to sacrifice some overall predictive accuracy by identifying a number of patients as being at risk who later do not show MAB (relatively poor specificity). Analyses here on the BRQ and PMQ found that missing data on the patient-completed questionnaires did not appear substantially to adversely affect the predictive characteristics of the patient-completed measures, which is helpful information to clinicians as incomplete data is not uncommon when tools are used in the clinic.
Here the BRQ showed relatively high sensitivity but low specificity (as it did in the original validation study).13This indicates that it tends to over-predict risk and the chance of MAB relative to other patient-completed risk assessment tools. This is likely due to the nature of the questionnaire. The BRQ is designed to assess multiple sources of possible risk and places a patient in a higher risk category if possible risk is identified in any of the twelve domains covered by its items. By using a wide domain of risk item content, the measure likely identifies more possible risk. Thus, if there seems to be any possible problem or risk area, the BRQ scoring usually places the patient at higher risk. Statistically, the BRQ is in many ways the opposite of other patient-completed risk assessment tools to which it has been compared (ORT, SOAPP-R, and PMQ). While other tools have relatively lower sensitivity and higher specificity, the BRQ in contrast has higher sensitivity and lower specificity. All of the patient-completed risk assessment tools have roughly the same overall predictive accuracy but the BRQ tends to err in a different way. Clinicians will need to be aware of this and use a patient-completed risk assessment tool with a conscious decision about whether they desire to err in identifying too many patients at risk for MAB or not identifying enough patients who may be at risk for MAB.
There are several methodological limitations in this current study. First, this study involved patients at a single pain practice, the same practice at which the first validation study was undertaken. The BRQ needs to be studied in other pain patient populations to assess its predictive accuracy in other situations. Additionally, this study’s population was almost entirely Caucasian. Studies in patient populations that are more diverse are needed.
The study was also conducted within a clinical setting in which the results of the risk assessment were used to alter the treatment plan to decrease the chances of medication aberrant behavior (through choices of medication and increased monitoring). Thus, the treatment plan actively worked to decrease the incidence of medication aberrant behavior in higher risk patients. This then decreased the likelihood of a “correct” prediction of future medication aberrant behavior. This is particularly true as some patients who were assessed as “Very High Risk” were not treated with opioids at all and were not included in this study as they had no opportunity to show medication aberrant behavior. This factor impacted the predictive accuracy of all the risk assessment tools here. In lieu of risk assessment research in which the results of the assessment are not known or used in clinical decision-making (a truly blinded study), the predictive accuracy results found here should be seen as relative among the tools used and not an absolute value of any tool’s predictive accuracy.
Prediction of medication aberrant behavior by pain patients on COT is still not as good as one would like, no matter what opioid risk tool is used. It is not clear if more work and study will come up with better assessment tools or if there will never be a good method or test that can predict this behavior at a high level accuracy as human behavior in general is so hard to predict. At this time those in the field of pain medicine who prescribe opioids for chronic pain conditions will have only these helpful but not perfect assessment tools for their use.
The Brief Risk Questionnaire (BRQ) © is a copyrighted assessment measure. The measure and its full scoring system are available from the first author at the website www. tedjonesresearch.com. Providers may download the measure and its scoring key without charge from this site for unlimited use. © Ted Jones Research, PLLC. The Brief Risk Questionnaire (“BRQ”) was created by Ted Jones, Ph.D. Unauthorized use, copying, and distribution of the BRQ in any form or manner, including without limitation posting on the internet, are strictly prohibited. This copy of the BRQ has been made available for use pursuant to the license agreement between the user and Ted Jones Research, PLLC. Users of the BRQ may not alter or modify the BRQ (including this Notice) in any fashion, through additions, deletions, or otherwise, without written consent. Further information, instructions for use of the BRQ, licenses for use, and requests for permissions are available by sending an email inquiry to tedjones@comcast.net or going to www.tedjonesresearch.com.
Table 3: Overall Predictive Results of the Study’s Four Risk Tools, Omitting Any Patients with Any Missing Data.
Sensitivity | Specificity | |
BRQ (N=124) | 75% | 38% |
ORT (N=142) | 25% | 83% |
PMQ (N=106) | 38% | 87% |
BRI (N=142) | 69% | 45% |
Table 4: Low, Medium and High Risk Categories of the BRQ.
Risk Category | Medication Aberrant Behavior Absent | Medication Aberrant Behavior Present | Total |
Low (0-2 points) | 38 (40%)* | 13 (27%) | 51 |
Medium (3-8 points) | 47 (50%) | 28 (58%) | 75 |
High (9+ points) | 9 (10%) | 7 (15%) | 16 |
94 | 48 | 142 |
*Column percentages