This week I focused on adding in the features of the viewer. This meant building in the back-end for features included the options menu: Summary (toggle on/off), Annotations (toggle on/off), Picture toolbar (toggle on/off), Full Screen, Download All, and Revert to Default.

If you click on “Options Menu” in the upper right corner, a drop down menu appears with gray highlighted options.

If you click on “Summary,” it will be toggled on. The clinician can then start typing in a summary for the current image. It is a pop-up window so it can be moved around for convenience so it doesn’t cover anything important on the screen while allowing the image to be as large as possible (although zooming in on the image isn’t built in yet).

For a doctor going back and reviewing the notes on a patient’s images, it would be very cumbersome to require him/her to keep opening up the summary pop-up window for each new image. Hence, if you embed the summary into the viewer then it can just sit below the image and the you can browse through all the media and their summaries quickly. The summary is still editable in this view and can be popped out of the main window at any time.

The toolbar can be switched off so that the doctor can use the entire screen space to view the image (not really useful until we have the zoom feature… but we can have more space to drag the image around the screen for now).

Full screen mode! You can’t really tell from the screenshot..but it is!

Really easy annotation tool. Just click anywhere on the image and start typing!

As soon as you hit save, the annotation will be saved and appear at that location on the image. If you drag the image around, it will still reappear at the same location on the image.

At any time, if annotations get in the way, they can be turned off by toggling “Annotations” off under the options menu. And wa-la! They’re gone. Until you want them again.

In this first week of the Google Summer of Code internship with OpenMRS, I created a basic module in OpenMRS and embedded a simple Flash application. The Flash app has the simple framework for the media viewer (patient name, date, image, and toolbar). The toolbar allows you to browse through all the thumbnails on the bottom and it also allows you to rotate images.

My Flash app embedded in OpenMRS.

If you scroll down, you can see the picture toolbar:

Rotating the image (pretty nifty animation with it too!)

The next step in coming up with a Moca OpenMRS media viewer was to get feedback from actual users. So we went straight to the 2 doctors on our team. We met and gathered feedback on my previous design by discussing things such as: What is the acutal process of diagnosing a patient from an image? What is involved in looking at radiology images? What is the difference between the clinician and radiologists’ role? Their advice was great and I also got input from a software/usability standpoint from the Moca developers. Below is the second iteration of a powerpoint mock-up of the media viewer for OpenMRS and Moca.

After this, we got more feedback from the doctors on our team and were able to get rid of features that weren’t as important to improve the simplicity of the interface. Another change was to collapse all the options across the top right corner of the viewer into an options menu that would also be scalable as I come up with more features that I want to include. The options menu also nicely handles the toggling on and off of the summary, annotations, and toolbar in the viewer.

This semester, I took the graduate course 6.344 Digital Image Processing. Our final project was to apply the image processing techniques we learned that semester to an interesting problem. It was pretty open-ended, so I decided to do a project that would also be useful for Moca: intelligent data capture on phones of x-ray images.

Background: One application for Moca is for the healthcare workers to capture images of the patient’s chest x-rays to detect tuberculosis, so that they can be sent to the radiologist in the city hospital for review. Otherwise, the paper x-ray would have to be physically trucked over to the hospital, which only occurs after enough x-rays pile enough to make the expensive trip worth it. The key, though, is for the healthcare worker to take a high enough quality image for the radiologist to be able to diagnose the patient properly for tuberculosis or not.

However, because healthcare workers aren’t trained to be experts, sometimes they naively take a picture that is not too small, blurred, under or over exposed, or they might cut off a portion of the image such as the written information on the x-ray. (This is reported by radiologists who have worked in the United States and in the Philippines). This makes the x-ray unreadable but by the time the problem in the image is seen by the radiologist, the patient has long left the rural health clinic and it’s not possible to take the image again.

Hence, we need a way to intelligently assess the quality of the image right when it is captured on the phone by the healthcare worker. We want to be able to tell them immediately if the image is (1) cut off or rotated, (2) under or over exposed, or (3) blurred. If it is, the application running on the phone would recommend them to immediately retake the image. In order to test these algorithms properly, I will use a set of x-ray images taken by a Google Android cell phone and provided to me by the University of Philippines National Telehealth Center.

Image Blurred

Image Cut Off

Image under exposed

As a result of this project, with the help of Moca team members, I was able to come up with a Matlab GUI that took as input an x-ray image and then output a result to a healthcare worker that determined whether the image of the x-ray was good enough quality to go onto a radiologist or if the healthcare worker should take the image again because of 1 of 3 reasons. When we actually integrate this with Moca, it will basically sit on the Android phone, wait for the healthcare worker to take an image of the x-ray with the phone, do some image processing, and then return a recommendation to the healthcare worker right then and there.

I am ultimately trying to redesign the way Moca allows doctors to view patient data (images, audio, video) with OpenMRS - the electronic medical record system. In order to do this, I am going to create a generic viewer for media files, for general use throughout OpenMRS. The vision is for any user of OpenMRS to click on a media file associated with a patient and then have it launch in the Flash viewer that I create, where the clinician would be able to review and annotate the file, and have it stored in the patient record for later reference.

Then I will proceed to integrate the viewer into the Moca platform. This means that when a clinician using Moca logs onto OpenMRS, he will see a queue of pending patient data files that require diagnosis (already built into Moca). If he clicks to view a particular patient’s visit, the media viewer will launch. On the left of the screen will be the “Moca panel” which holds patient responses to pertinent questions (How bad is the pain? When did you get surgery?) and input fields for making a treatment and diagnosis recommendation. On the right hand side of the screen will be the OpenMRS Flash viewer that I mentioned above. The clinician will be able to browse all images, video, and audio files of that patient from that same visit, manipulate the image (zoom, increase contrast, brighten, sharpen), annotate them with his interpretations, enter in his diagnosis/treatment recommendation for the patient, and then hit a button to immediately send an SMS with his response back to the healthcare worker who originally interacted with the patient.

To start off, I came up with a mock-up of what the viewer would look like in OpenMRS to handle (1) images, (2) audio, and (3) videos.

On the server side backend, Moca is fully integrated with an OpenMRS - an open source electronic medical record system for developing countries.

“OpenMRS^® is a community-developed, open-source, enterprise electronic medical record system framework. We’ve come together to specifically respond to those actively building and managing health systems in the developing world, where AIDS, tuberculosis, and malaria afflict the lives of millions. Our mission is to foster self-sustaining health information technology implementations in these environments through peer mentorship, proactive collaboration, and a code base that equals or surpasses proprietary equivalents.”

Learn more here: http://openmrs.org/wiki/OpenMRS

This summer, Jenny Liu and myself (Katherine Kuan) from the Moca team will be working as interns for OpenMRS through Google Summer of Code 2009. While we develop more sophisticated features for Moca on the OpenMRS side this summer, we will design them to be reusable modules of source code to contribute to the OpenMRS central repository. This way, other implementers/developers will be able to use these features (audio/video upload and review as well as OpenMRS patient authentication via hybrid online/offline mobile phones).

Specifically, I will be working on a media viewer module in OpenMRS that includes video upload and review from mobile phones. As part of the Moca team, we would like to add support for video upload and review within OpenMRS and on the Google Android platform, initially for ultrasound data review and diagnosis. By summer 2009 we should have a Java code-base for reading DICOM videos from most ultrasound devices and compressing them for upload. We should also have a prototype for uploading this data to OpenMRS. This project aims to knit together these algorithms onto the Android platform, and extend the OpenMRS interface to allow audio and video review and annotation. Video such as ultrasound will be delivered into OpenMRS via mobile phones. Finally, we are interested in integrating open source image segmentation and classification algorithms into the software (either on the server or the client). I will track my progress on this blog as the summer goes by, under the tag GSOC (Google Summer of Code) Media Viewer.