This week I continued to work on integrating the media viewer with OpenMRS through HTTP requests. The basic structure is for OpenMRS to pass an encounter ID (unique to each patient throughout the entire instance of OpenMRS running on a server) to Flex. Then Flex will do an HTTP get with that encounter ID to a Java encounter servlet. The servlet will write an XML document in response (see previous blog entry) that describes the patient’s name, ID, media files, and respective annotations/summaries. This works so far. Then Flex will build up an Actionscript encounter object with these media files. For each media file, it will do an HTTP get with that obs ID to a Java complex obs servlet. The servlet should send back the image/audio/media file. I’m working on this so far, and the problem is to figure out how to load the image file in Flex with URLLoader, Loader, or some other class.

I also worked on creating a VM image of the Moca server to bring to the Philippines. I downloaded VMware Workstation, installed Ubuntu 9.04 on top of it, and am currently in the process of installing OpenMRS. After this, I will have to load in the Moca module into OpenMRS and then install the Moca Dispatch Server on it. I’m creating a document that will track this process step-by-step and will post it here when I’m done!

In addition, I’m leaving for the Philippines in 3 days, so I’m also doing some last-minute research on telemedicine, eHealth, EMRs, and eLearning. I plan to summarize the research I perform into executive summaries that can be presented to the stakeholders of a telemedicine business proposal for the country - PhilHealth Insurance Coproration and Department of Health to name a few. Below are some papers that I’ll be reading:

An information system and medical record to support HIV treatment in rural Haiti. Fraser HSF, Jazayeri D, Nevil P, Karacaoglu Y, Farmer PE, Lyon E Smith-Fawzi MK, Leandre F, Choi S, Mukherjee JS. British Medical Journal, 2004

Diagnostic Accuracy of Chest X-rays Acquired Using a Digital Camera for Low-Cost Teleradiology. Szot A, Jacobson F, Munn S, Jazayeri D, Nardell E, Harrison D Drosten R, Ohno-Machado L, Smeaton LM, Fraser HSF. . Int. J. Med. Inform., 2004

A web-based laboratory information system to improve quality of care of tuberculosis patients in Peru: functional requirements, implementation and usage statistics. Joaquin A Blaya , Sonya S Shin , Martin JA Yagui , Gloria Yale, Carmen Z Suarez , Luis L Asencios , J Peter Cegielski  and Hamish S Fraser. BMC Medical Informatics and Decision Making 2007

Medical information systems: A foundation for healthcare technologies in developing countries. Gari D Clifford, Joaquin A Blaya, Rachel Hall-Clifford, Hamish SF Fraser, BioMedical Engineering OnLin,  2008

Design of Health Care Technologies for the Developing World. Robert A. Malkin,  Annu. Rev. Biomed. Eng. 2007

On May 26, 2009, the OpenMRS Implementers and Developers Meeting took place in Boston at a new Research Building at Harvard Medical School. Clark, Gari, Leo, Zack, and I attended on behalf of the Moca team. We met many interesting people who were involved in various aspects of OpenMRS - founders, developers, implementers in the field in Africa, university researchers, hospital staff, and company representatives.

The day opened with an introduction of OpenMRS and its vision by co-founder Hamish Fraser. (See slides posted here: http://www.slideshare.net/hamishfraser/openmrs-meeting-boston-introduction). We also had presentations on the current capabilities of OpenMRS as well as deployments (in Kenya, Rwanda, South Africa, Zimbabwe, and Mozambique). In the afternoon, we had breakout sessions about OpenMRS deployment in the field, OpenMRS API and module development, standards/integration of OpenMRS, and clinical decision support. Overall, the meeting was very productive and many sources of collaboration were identified among all the people involved. For example, for myself, I was able to meet many OpenMRS developers for help on my GSOC project, as well as experienced implementers of OpenMRS (for help on research with my research fellowship in the Philippines this summer).

To prepare for my trip to the Philippines in a couple of weeks, I also had a conference call with the University of Philippines team about technical requirements for the pilot and the workflow of the pilot. In addition, I also worked on learning how to setup/install OpenMRS, Tomcat, MySQL so that I would be able to help get the technical team in the Philippines get Moca up and running.

As for my GSOC project, this week I worked on integrating my Flash application with OpenMRS. I made an encounter servlet in OpenMRS to pass encounter information (patient name, ID, image/audio/video obs IDs, and dates) to the Flash application. It didn’t quite work though, so i backed up and tried something simplier. I managed to get successful HTTP get/post requests of text data from Flex to OpenMRS working. From my understanding, passing a Java object into Flex to be converted into an AS3 object is done with Flash remoting, which is a proprietary package. After examining the problem further, it seems that we don’t need the full object to be passed into Flex and text would work just as well. Hence I created an XML schema with a parser for passing encounter data between Flash and Java:

<?xml version=”1.0″ encoding=”utf-8″?>
<encounter patientID=”1010″ patientFirst=”John” patientLast=”Smith”>
<genericFile type=”image” obsID=”39″ summary=”This x-ray is abnormal. Patient could have TB.”>
<annotation text=”This appears abnormal” x=”10″ y=”10″/>
<annotation text=”This also is abnormal” x=”20″ y=”20″/>
<annotation text=”This could be TB” x=”30″ y=”30″/>
</genericFile>
<genericFile type=”audio” obsID=”45″ summary=”The x-ray appears normal.”>
<annotation text=”This is normal” x=”10″ y=”10″/>
</genericFile>
</encounter>

This week I focused on adding in the features of the viewer. This meant building in the back-end for features included the options menu: Summary (toggle on/off), Annotations (toggle on/off), Picture toolbar (toggle on/off), Full Screen, Download All, and Revert to Default.

If you click on “Options Menu” in the upper right corner, a drop down menu appears with gray highlighted options.

If you click on “Summary,” it will be toggled on. The clinician can then start typing in a summary for the current image. It is a pop-up window so it can be moved around for convenience so it doesn’t cover anything important on the screen while allowing the image to be as large as possible (although zooming in on the image isn’t built in yet).

For a doctor going back and reviewing the notes on a patient’s images, it would be very cumbersome to require him/her to keep opening up the summary pop-up window for each new image. Hence, if you embed the summary into the viewer then it can just sit below the image and the you can browse through all the media and their summaries quickly. The summary is still editable in this view and can be popped out of the main window at any time.

The toolbar can be switched off so that the doctor can use the entire screen space to view the image (not really useful until we have the zoom feature… but we can have more space to drag the image around the screen for now).

Full screen mode! You can’t really tell from the screenshot..but it is!

Really easy annotation tool. Just click anywhere on the image and start typing!

As soon as you hit save, the annotation will be saved and appear at that location on the image. If you drag the image around, it will still reappear at the same location on the image.

At any time, if annotations get in the way, they can be turned off by toggling “Annotations” off under the options menu. And wa-la! They’re gone. Until you want them again.

In this first week of the Google Summer of Code internship with OpenMRS, I created a basic module in OpenMRS and embedded a simple Flash application. The Flash app has the simple framework for the media viewer (patient name, date, image, and toolbar). The toolbar allows you to browse through all the thumbnails on the bottom and it also allows you to rotate images.

My Flash app embedded in OpenMRS.

If you scroll down, you can see the picture toolbar:

Rotating the image (pretty nifty animation with it too!)

The next step in coming up with a Moca OpenMRS media viewer was to get feedback from actual users. So we went straight to the 2 doctors on our team. We met and gathered feedback on my previous design by discussing things such as: What is the acutal process of diagnosing a patient from an image? What is involved in looking at radiology images? What is the difference between the clinician and radiologists’ role? Their advice was great and I also got input from a software/usability standpoint from the Moca developers. Below is the second iteration of a powerpoint mock-up of the media viewer for OpenMRS and Moca.

After this, we got more feedback from the doctors on our team and were able to get rid of features that weren’t as important to improve the simplicity of the interface. Another change was to collapse all the options across the top right corner of the viewer into an options menu that would also be scalable as I come up with more features that I want to include. The options menu also nicely handles the toggling on and off of the summary, annotations, and toolbar in the viewer.

This semester, I took the graduate course 6.344 Digital Image Processing. Our final project was to apply the image processing techniques we learned that semester to an interesting problem. It was pretty open-ended, so I decided to do a project that would also be useful for Moca: intelligent data capture on phones of x-ray images.

Background: One application for Moca is for the healthcare workers to capture images of the patient’s chest x-rays to detect tuberculosis, so that they can be sent to the radiologist in the city hospital for review. Otherwise, the paper x-ray would have to be physically trucked over to the hospital, which only occurs after enough x-rays pile enough to make the expensive trip worth it. The key, though, is for the healthcare worker to take a high enough quality image for the radiologist to be able to diagnose the patient properly for tuberculosis or not.

However, because healthcare workers aren’t trained to be experts, sometimes they naively take a picture that is not too small, blurred, under or over exposed, or they might cut off a portion of the image such as the written information on the x-ray. (This is reported by radiologists who have worked in the United States and in the Philippines). This makes the x-ray unreadable but by the time the problem in the image is seen by the radiologist, the patient has long left the rural health clinic and it’s not possible to take the image again.

Hence, we need a way to intelligently assess the quality of the image right when it is captured on the phone by the healthcare worker. We want to be able to tell them immediately if the image is (1) cut off or rotated, (2) under or over exposed, or (3) blurred. If it is, the application running on the phone would recommend them to immediately retake the image. In order to test these algorithms properly, I will use a set of x-ray images taken by a Google Android cell phone and provided to me by the University of Philippines National Telehealth Center.

Image Blurred

Image Cut Off

Image under exposed

As a result of this project, with the help of Moca team members, I was able to come up with a Matlab GUI that took as input an x-ray image and then output a result to a healthcare worker that determined whether the image of the x-ray was good enough quality to go onto a radiologist or if the healthcare worker should take the image again because of 1 of 3 reasons. When we actually integrate this with Moca, it will basically sit on the Android phone, wait for the healthcare worker to take an image of the x-ray with the phone, do some image processing, and then return a recommendation to the healthcare worker right then and there.

I am ultimately trying to redesign the way Moca allows doctors to view patient data (images, audio, video) with OpenMRS - the electronic medical record system. In order to do this, I am going to create a generic viewer for media files, for general use throughout OpenMRS. The vision is for any user of OpenMRS to click on a media file associated with a patient and then have it launch in the Flash viewer that I create, where the clinician would be able to review and annotate the file, and have it stored in the patient record for later reference.

Then I will proceed to integrate the viewer into the Moca platform. This means that when a clinician using Moca logs onto OpenMRS, he will see a queue of pending patient data files that require diagnosis (already built into Moca). If he clicks to view a particular patient’s visit, the media viewer will launch. On the left of the screen will be the “Moca panel” which holds patient responses to pertinent questions (How bad is the pain? When did you get surgery?) and input fields for making a treatment and diagnosis recommendation. On the right hand side of the screen will be the OpenMRS Flash viewer that I mentioned above. The clinician will be able to browse all images, video, and audio files of that patient from that same visit, manipulate the image (zoom, increase contrast, brighten, sharpen), annotate them with his interpretations, enter in his diagnosis/treatment recommendation for the patient, and then hit a button to immediately send an SMS with his response back to the healthcare worker who originally interacted with the patient.

To start off, I came up with a mock-up of what the viewer would look like in OpenMRS to handle (1) images, (2) audio, and (3) videos.

On the server side backend, Moca is fully integrated with an OpenMRS - an open source electronic medical record system for developing countries.

“OpenMRS^® is a community-developed, open-source, enterprise electronic medical record system framework. We’ve come together to specifically respond to those actively building and managing health systems in the developing world, where AIDS, tuberculosis, and malaria afflict the lives of millions. Our mission is to foster self-sustaining health information technology implementations in these environments through peer mentorship, proactive collaboration, and a code base that equals or surpasses proprietary equivalents.”

Learn more here: http://openmrs.org/wiki/OpenMRS

This summer, Jenny Liu and myself (Katherine Kuan) from the Moca team will be working as interns for OpenMRS through Google Summer of Code 2009. While we develop more sophisticated features for Moca on the OpenMRS side this summer, we will design them to be reusable modules of source code to contribute to the OpenMRS central repository. This way, other implementers/developers will be able to use these features (audio/video upload and review as well as OpenMRS patient authentication via hybrid online/offline mobile phones).

Specifically, I will be working on a media viewer module in OpenMRS that includes video upload and review from mobile phones. As part of the Moca team, we would like to add support for video upload and review within OpenMRS and on the Google Android platform, initially for ultrasound data review and diagnosis. By summer 2009 we should have a Java code-base for reading DICOM videos from most ultrasound devices and compressing them for upload. We should also have a prototype for uploading this data to OpenMRS. This project aims to knit together these algorithms onto the Android platform, and extend the OpenMRS interface to allow audio and video review and annotation. Video such as ultrasound will be delivered into OpenMRS via mobile phones. Finally, we are interested in integrating open source image segmentation and classification algorithms into the software (either on the server or the client). I will track my progress on this blog as the summer goes by, under the tag GSOC (Google Summer of Code) Media Viewer.