International Journal of Clinical Anesthesiology

See One, Do One, Take an Assessment

Editorial | Open Access

  • 1. Department of Anesthesiology and Perioperative Medicine and the Department of Radiology, Oregon Health and Science University, Portland, Oregon
  • 2. Associate Director, Regional Anesthesia and Acute Pain Fellowship, University of Miami, Miller School of Medicine, Miami, Florida USA
+ Show More - Show Less
Corresponding Authors
Glenn Woodworth, Department of Anesthesiology and Perioperative Medicine and the Department of Radiology, Oregon Health and Science University, Portland, Oregon

 Woodworth G, Missair A (2013) See One, Do One, Take an Assessment. Int J Clin Anesthesiol 1: 1001.


Medicine has a long history that is fundamentally rooted in an apprenticeship model of education. A common refrain heard during residency has been “see one, do one, teach one.” The new generation of physicians is likely to hear a much different tune. Medical Education is rapidly evolving from the traditional experienced-based model to a competency-based model with defined educational goals and competency requirements [1,2]. Recognizing the importance of competency assessment and certification for specialists, the American Board of Anesthesiology administered its first written examination in 1937 [3]. The exam covered pharmacology, pathology, anatomy, physics and chemistry. Examinees were required to complete 3 of 5 essay questions within the allotted time. This exam eventually evolved into a multiple choice assessment of medical knowledge. Medicine and graduate medical education have a time-honored tradition of assessing medical knowledge via this model and have led to the graduation of many medical students, who were considered to be test-taking machines.

Of course, physician competency requires more than medical knowledge. The last 15 years have seen dramatic changes in the definition and assessment of physician competency. In the United States this was reflected in the cooperation between the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) to define and assess competency [4]. In 1998 the ACGME, with input from the ABMS, began a 15-year “Outcome Project” to revise the process of accreditation for U.S. training programs. The goal of the project was to realign graduate medical education with academic achievement as well as patient outcomes. In the era of evidence-based medicine, the project sought to answer the question, “How can we demonstrate that educational programs are effective?” This goal requires an established definition of physician competence as well as methods for the assessment of the attainment of competence during training. In 2002, Epstein published a landmark article in the Journal of the American Medical Association, proposing that physician competence requires knowledge, communication, reasoning, technical skills, values and reflection [5]. At about the same time, the ACGME adopted six competency domains to assess medical residents: medical knowledge, patient care, professionalism, communication and interpersonal skills, practice-based learning and improvement, and systems-based practice [6]. In 2009, the ACGME updated the Outcome Project to include milestones. Each residency review committee (RRC) was tasked to develop specific competencybased milestones for their specialties. Implementation of milestones by anesthesia residencies is scheduled for 2014. The ACGME is encouraging individual specialties and programs to innovate and develop their own tools to determine if specific competency milestones have been met [7,8]. To assist in this effort the ACGME provided a list of potential types of assessment tools to consider implementing including Objective Structured Clinical Exams (OSCE), patient surveys, portfolios, record review and simulation.

To be a truly valid assessment of competency, the assessment tool must have been developed using psychometric principles and proven to be valid and reliable [9-13]. Few programs have the resources and expertise to develop valid assessments. Admittedly, assessments of residency milestones are not highstakes examinations; however, it would be preferential to use validated assessments. Lurie reviewed the existing medical literature and found few published assessment tools to be reliable and valid [14]. In the United States the National Board of Medical Examiners implemented simulation-based clinical skills assessment in 2005 [15]. Recently, attention has focused on how best to assess other important aspects of competency including judgment, teamwork, communication, professionalism, and procedural skills [10,11]. Anesthesiology has made some initial progress in the development of assessment tools. Early forms of assessment in anesthesia focused on knowledge or the application of knowledge in the form of recall of facts or principles, in part because they could be readily assessed with multiple choice style exams. Investigators are now testing the application of a variety of assessment tool types in anesthesiology including high and low fidelity simulation, OSCE, and Objective Structured Assessment of Technical Skills (OSATS) [16-25]. One of the most interesting implementations has been the incorporation of simulation and OSCE style assessments into the Israeli national board examination process [26]. The American Board of Anesthesiology has announced the restructuring of the Part 2 Oral Exam into an “Applied Exam” that will include OSCE style assessments [27].

No matter what instrument is used for assessment of milestones, it is necessary to use appropriate techniques to determine passing scores [28]. This is critically important for high-stakes examinations, but is also important for milestone assessment. Few of the published assessment tools in anesthesia address this concept.

The implementation of milestone assessments in a residency program must be practical. Given the large number of potential milestones and the variety of assessments that will be required, administration of the assessments could be problematic. In our institutions, we have looked at the regional anesthesia curriculum as a pilot for implementation of milestone assessments. For example, one aspect of regional anesthesia that will require assessment is competency with procedures. Is it practical, or necessary, to assess resident competency for each and every type of peripheral nerve block? We have chosen to pilot an assessment approach that looks at procedural competency and competency in component skills. In order to assess competency with peripheral nerve blocks, we are pilot testing a modification of an existing validated OSCE/OSATS for interscalene blocks [20]. The assessment tool can be used with a patient simulator or while a resident is performing the block on an actual patient. The tool consists of a checklist and global rating scales covering everything from equipment set-up and adherence to sterile technique, to probe handling and proper positioning of the nerve target on the ultrasound screen. Because it may not be practical to administer this type of assessment for every block type, we have chosen to look at the component skills of ultrasound-guided nerve blockade. In addition to sterile technique, ergonomics, probe handling, and nerve target acquisition with ultrasound, the resident must be able to recognize and interpret ultrasound images and be skillful at guiding needles to targets under ultrasound. If the resident can pass the basic interscalene block OSCE/OSATS, we assume they are competent in sterile technique, ergonomics, etc. for other blocks. However, in addition to the interscalene block assessment, we will require residents to pass a needle-guidance under ultrasound assessment, and an ultrasound interpretation skills assessment, to demonstrate competency in ultrasound-guided peripheral nerve blocks. Despite the importance of ultrasound to regional anesthesia, a validated assessment of ultrasound interpretation skills has not been published. Using sound psychometric principals we have developed an ultrasound interpretation skills assessment that requires learners to view ultrasound clips and answer questions about the clips. The clips are obtained from real patients, have varying degrees of image clarity as in real-life, and cover a defined knowledgebase (content validity). The questions have been designed to test a range of competency, with some questions that should be answered correctly by a junior resident, and others that should be answered correctly by a graduating senior resident. The stem for each question has an illustrated diagram and textual description of how the clip was obtained. The stem may ask the learner to identify a structure, select how to move the probe to better visualize a structure, or ask the learner why the probe should be repositioned. The exam has been pilot tested and is currently undergoing a validation phase across multiple institutions that will include determination of passing scores.

In this new era of medicine, we can expect increased scrutiny over competency at every level, including for licensure, during residency, for certification and maintenance of certification, for credentialing with hospitals and health plans, and for the adoption of new techniques or technologies that arrive on the scene after we have finished training. The role of the research community should be to provide practical and validated assessments to satisfy these demands. As academicians training future physicians, we must embrace the responsibility that comes with mentorship and the need to validate the models and benchmarks used to assess our future peers. In return, our specialties will safeguard the quality of their trainees and the safety of our patients.


1. Aggarwal R, Darzi A. Technical-skills training in the 21st century. N Engl J Med. 2006; 355: 2695-2696.

2. Reznick RK, MacRae H. Teaching surgical skills--changes in the wind. N Engl J Med. 2006; 355: 2664-2669.

3. Bacon DR, Lema MJ. To define a specialty: a brief history of the American Board of Anesthesiology’s first written examination. J Clin Anesth. 1992; 4: 489-497.

4. Accreditation Council for Graduate Medical Education (ACGME). 2001. The Project: Introduction.

5. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002; 287: 226-235.

6. Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff (Millwood). 2002; 21: 103-111.

7. Accreditation Council for Graduate Medical Education (ACGME). 2009. The next Accreditation System: Frequently Asked Questions.

8. Nasca TJ. The CEO’s first column – the next step in the OutcomesBased Accreditation Project. ACGME Bulletin. 2008; May: 2–4.

9. pstein RM. Assessment in medical education. N Engl J Med. 2007; 356: 387-396.

10. Boulet JR, Murray D. Review article: assessment in anesthesiology education. Can J Anaesth. 2012; 59: 182-192.

11. Bould MD, Crabtree NA, Naik VN. Assessment of procedural skills in anaesthesia. Br J Anaesth. 2009; 103: 472-483.

12. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006; 119: 166.

13. Gallagher AG, Ritter EM, Satava RM. Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc. 2003; 17: 1525-1529.

14. Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 2009; 84: 301-309.

15. apadakis MA. The Step 2 clinical-skills examination. N Engl J Med. 2004; 350: 1703-1705.

16. Ben-Menachem E, Ezri T, Ziv A, Sidi A, Brill S, Berkenstadt H. Objective Structured Clinical Examination-based assessment of regional anesthesia skills: the Israeli National Board Examination in Anesthesiology experience. Anesth Analg. 2011; 112: 242-5.

17. Berkenstadt H, Ziv A, Gafni N, Sidi A. The validation process of incorporating simulation-based accreditation into the anesthesiology Israeli national board exams. Isr Med Assoc J. 2006; 8: 728-733.

18. Breen D, Bogar L, Heigl P, Rittberger J, Shorten GD. Validation of a clinical assessment tool for spinal anaesthesia. Acta Anaesthesiol Scand. 2011; 55: 653-657.

19. dler AA. The use of simulation education in competency assessment: more questions than answers. Anesthesiology. 2008; 108: 167.

20. Naik VN, Perlas A, Chandra DB, Chung DY, Chan VW. An assessment tool for brachial plexus regional anesthesia performance: establishing construct validity and reliability. Reg Anesth Pain Med. 2007; 32: 41- 45.

21. Ringsted C, Østergaard D, Ravn L, Pedersen JA, Berlac PA, van der Vleuten CP. A feasibility study comparing checklists and global rating forms to assess resident performance in clinical skills. Med Teach. 2003; 25: 654-658.

22. Ringsted C, Østergaard D, Scherpbier A. Embracing the new paradigm of assessment in residency training: an assessment programme for first-year residency training in anaesthesiology. Med Teach. 2003; 25: 54-62.

23. Schwid HA, Rooke GA, Carline J, et al. Anesthesia Simulator Research Consortium. Evaluation of anesthesia residents using mannequinbased simulation: a multi-institutional study. Anesthesiology. 2002; 97:1434-1444.

24. etzlaff JE. Assessment of competency in anesthesiology. Anesthesiology. 2007; 106: 812-825.

25. Wisborg T, Ringsted C. Tools for the assessment of practical skills and performance. Acta Anaesthesiol Scand. 2011; 55: 633-634.

26. Ben-Menachem E, Ezri T, Ziv A, Sidi A, Brill S, Berkenstadt H. Objective Structured Clinical Examination-based assessment of regional anesthesia skills: the Israeli National Board Examination in Anesthesiology experience. Anesth Analg. 2011; 112: 242-5.

27. http://www.theaba.org/Home/TrainingPrograms. Accessed 7/24/13.

28. Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med. 2006; 18: 50-57.

Received : 15 Jul 2013
Accepted : 17 Jul 2013
Published : 20 Aug 2013
Annals of Otolaryngology and Rhinology
ISSN : 2379-948X
Launched : 2014
JSM Schizophrenia
Launched : 2016
Journal of Nausea
Launched : 2020
JSM Internal Medicine
Launched : 2016
JSM Hepatitis
Launched : 2016
JSM Oro Facial Surgeries
ISSN : 2578-3211
Launched : 2016
Journal of Human Nutrition and Food Science
ISSN : 2333-6706
Launched : 2013
JSM Regenerative Medicine and Bioengineering
ISSN : 2379-0490
Launched : 2013
JSM Spine
ISSN : 2578-3181
Launched : 2016
Archives of Palliative Care
ISSN : 2573-1165
Launched : 2016
JSM Nutritional Disorders
ISSN : 2578-3203
Launched : 2017
Annals of Neurodegenerative Disorders
ISSN : 2476-2032
Launched : 2016
Journal of Fever
ISSN : 2641-7782
Launched : 2017
JSM Bone Marrow Research
ISSN : 2578-3351
Launched : 2016
JSM Mathematics and Statistics
ISSN : 2578-3173
Launched : 2014
Journal of Autoimmunity and Research
ISSN : 2573-1173
Launched : 2014
JSM Arthritis
ISSN : 2475-9155
Launched : 2016
JSM Head and Neck Cancer-Cases and Reviews
ISSN : 2573-1610
Launched : 2016
JSM General Surgery Cases and Images
ISSN : 2573-1564
Launched : 2016
JSM Anatomy and Physiology
ISSN : 2573-1262
Launched : 2016
JSM Dental Surgery
ISSN : 2573-1548
Launched : 2016
Annals of Emergency Surgery
ISSN : 2573-1017
Launched : 2016
Annals of Mens Health and Wellness
ISSN : 2641-7707
Launched : 2017
Journal of Preventive Medicine and Health Care
ISSN : 2576-0084
Launched : 2018
Journal of Chronic Diseases and Management
ISSN : 2573-1300
Launched : 2016
Annals of Vaccines and Immunization
ISSN : 2378-9379
Launched : 2014
JSM Heart Surgery Cases and Images
ISSN : 2578-3157
Launched : 2016
Annals of Reproductive Medicine and Treatment
ISSN : 2573-1092
Launched : 2016
JSM Brain Science
ISSN : 2573-1289
Launched : 2016
JSM Biomarkers
ISSN : 2578-3815
Launched : 2014
JSM Biology
ISSN : 2475-9392
Launched : 2016
Archives of Stem Cell and Research
ISSN : 2578-3580
Launched : 2014
Annals of Clinical and Medical Microbiology
ISSN : 2578-3629
Launched : 2014
JSM Pediatric Surgery
ISSN : 2578-3149
Launched : 2017
Journal of Memory Disorder and Rehabilitation
ISSN : 2578-319X
Launched : 2016
JSM Tropical Medicine and Research
ISSN : 2578-3165
Launched : 2016
JSM Head and Face Medicine
ISSN : 2578-3793
Launched : 2016
JSM Cardiothoracic Surgery
ISSN : 2573-1297
Launched : 2016
JSM Bone and Joint Diseases
ISSN : 2578-3351
Launched : 2017
JSM Bioavailability and Bioequivalence
ISSN : 2641-7812
Launched : 2017
JSM Atherosclerosis
ISSN : 2573-1270
Launched : 2016
Journal of Genitourinary Disorders
ISSN : 2641-7790
Launched : 2017
Journal of Fractures and Sprains
ISSN : 2578-3831
Launched : 2016
Journal of Autism and Epilepsy
ISSN : 2641-7774
Launched : 2016
Annals of Marine Biology and Research
ISSN : 2573-105X
Launched : 2014
JSM Health Education & Primary Health Care
ISSN : 2578-3777
Launched : 2016
JSM Communication Disorders
ISSN : 2578-3807
Launched : 2016
Annals of Musculoskeletal Disorders
ISSN : 2578-3599
Launched : 2016
Annals of Virology and Research
ISSN : 2573-1122
Launched : 2014
JSM Renal Medicine
ISSN : 2573-1637
Launched : 2016
Journal of Muscle Health
ISSN : 2578-3823
Launched : 2016
JSM Genetics and Genomics
ISSN : 2334-1823
Launched : 2013
JSM Anxiety and Depression
ISSN : 2475-9139
Launched : 2016
Clinical Journal of Heart Diseases
ISSN : 2641-7766
Launched : 2016
Annals of Medicinal Chemistry and Research
ISSN : 2378-9336
Launched : 2014
JSM Pain and Management
ISSN : 2578-3378
Launched : 2016
JSM Women's Health
ISSN : 2578-3696
Launched : 2016
Clinical Research in HIV or AIDS
ISSN : 2374-0094
Launched : 2013
Journal of Endocrinology, Diabetes and Obesity
ISSN : 2333-6692
Launched : 2013
Journal of Substance Abuse and Alcoholism
ISSN : 2373-9363
Launched : 2013
JSM Neurosurgery and Spine
ISSN : 2373-9479
Launched : 2013
Journal of Liver and Clinical Research
ISSN : 2379-0830
Launched : 2014
Journal of Drug Design and Research
ISSN : 2379-089X
Launched : 2014
JSM Clinical Oncology and Research
ISSN : 2373-938X
Launched : 2013
JSM Bioinformatics, Genomics and Proteomics
ISSN : 2576-1102
Launched : 2014
JSM Chemistry
ISSN : 2334-1831
Launched : 2013
Journal of Trauma and Care
ISSN : 2573-1246
Launched : 2014
JSM Surgical Oncology and Research
ISSN : 2578-3688
Launched : 2016
Annals of Food Processing and Preservation
ISSN : 2573-1033
Launched : 2016
Journal of Radiology and Radiation Therapy
ISSN : 2333-7095
Launched : 2013
JSM Physical Medicine and Rehabilitation
ISSN : 2578-3572
Launched : 2016
Annals of Clinical Pathology
ISSN : 2373-9282
Launched : 2013
Annals of Cardiovascular Diseases
ISSN : 2641-7731
Launched : 2016
Journal of Behavior
ISSN : 2576-0076
Launched : 2016
Annals of Clinical and Experimental Metabolism
ISSN : 2572-2492
Launched : 2016
Clinical Research in Infectious Diseases
ISSN : 2379-0636
Launched : 2013
JSM Microbiology
ISSN : 2333-6455
Launched : 2013
Journal of Urology and Research
ISSN : 2379-951X
Launched : 2014
Journal of Family Medicine and Community Health
ISSN : 2379-0547
Launched : 2013
Annals of Pregnancy and Care
ISSN : 2578-336X
Launched : 2017
JSM Cell and Developmental Biology
ISSN : 2379-061X
Launched : 2013
Annals of Aquaculture and Research
ISSN : 2379-0881
Launched : 2014
Clinical Research in Pulmonology
ISSN : 2333-6625
Launched : 2013
Journal of Immunology and Clinical Research
ISSN : 2333-6714
Launched : 2013
Annals of Forensic Research and Analysis
ISSN : 2378-9476
Launched : 2014
JSM Biochemistry and Molecular Biology
ISSN : 2333-7109
Launched : 2013
Annals of Breast Cancer Research
ISSN : 2641-7685
Launched : 2016
Annals of Gerontology and Geriatric Research
ISSN : 2378-9409
Launched : 2014
Journal of Sleep Medicine and Disorders
ISSN : 2379-0822
Launched : 2014
JSM Burns and Trauma
ISSN : 2475-9406
Launched : 2016
Chemical Engineering and Process Techniques
ISSN : 2333-6633
Launched : 2013
Annals of Clinical Cytology and Pathology
ISSN : 2475-9430
Launched : 2014
JSM Allergy and Asthma
ISSN : 2573-1254
Launched : 2016
Journal of Neurological Disorders and Stroke
ISSN : 2334-2307
Launched : 2013
Annals of Sports Medicine and Research
ISSN : 2379-0571
Launched : 2014
JSM Sexual Medicine
ISSN : 2578-3718
Launched : 2016
Annals of Vascular Medicine and Research
ISSN : 2378-9344
Launched : 2014
JSM Biotechnology and Biomedical Engineering
ISSN : 2333-7117
Launched : 2013
Journal of Hematology and Transfusion
ISSN : 2333-6684
Launched : 2013
JSM Environmental Science and Ecology
ISSN : 2333-7141
Launched : 2013
Journal of Cardiology and Clinical Research
ISSN : 2333-6676
Launched : 2013
JSM Nanotechnology and Nanomedicine
ISSN : 2334-1815
Launched : 2013
Journal of Ear, Nose and Throat Disorders
ISSN : 2475-9473
Launched : 2016
JSM Ophthalmology
ISSN : 2333-6447
Launched : 2013
Journal of Pharmacology and Clinical Toxicology
ISSN : 2333-7079
Launched : 2013
Annals of Psychiatry and Mental Health
ISSN : 2374-0124
Launched : 2013
Medical Journal of Obstetrics and Gynecology
ISSN : 2333-6439
Launched : 2013
Annals of Pediatrics and Child Health
ISSN : 2373-9312
Launched : 2013
JSM Clinical Pharmaceutics
ISSN : 2379-9498
Launched : 2014
JSM Foot and Ankle
ISSN : 2475-9112
Launched : 2016
JSM Alzheimer's Disease and Related Dementia
ISSN : 2378-9565
Launched : 2014
Journal of Addiction Medicine and Therapy
ISSN : 2333-665X
Launched : 2013
Journal of Veterinary Medicine and Research
ISSN : 2378-931X
Launched : 2013
Annals of Public Health and Research
ISSN : 2378-9328
Launched : 2014
Annals of Orthopedics and Rheumatology
ISSN : 2373-9290
Launched : 2013
Journal of Clinical Nephrology and Research
ISSN : 2379-0652
Launched : 2014
Annals of Community Medicine and Practice
ISSN : 2475-9465
Launched : 2014
Annals of Biometrics and Biostatistics
ISSN : 2374-0116
Launched : 2013
JSM Clinical Case Reports
ISSN : 2373-9819
Launched : 2013
Journal of Cancer Biology and Research
ISSN : 2373-9436
Launched : 2013
Journal of Surgery and Transplantation Science
ISSN : 2379-0911
Launched : 2013
Journal of Dermatology and Clinical Research
ISSN : 2373-9371
Launched : 2013
JSM Gastroenterology and Hepatology
ISSN : 2373-9487
Launched : 2013
Annals of Nursing and Practice
ISSN : 2379-9501
Launched : 2014
JSM Dentistry
ISSN : 2333-7133
Launched : 2013
Author Information X