DxGPT is a diagnostic decision support software based on GPT-4. GPT-4 is an Artificial Intelligence (AI) language model developed by OpenAI. It is designed to generate text from previous input. This means that the software can generate a list of diseases from a description of symptoms. However, due to the characteristics of the model, there is a possibility that the software has errors. Therefore, it should not be used for medical use.
Once the list of diseases is generated, it can be completed with further information to refine the diagnosis. This includes collecting clinical data, performing laboratory tests, and gathering information from the medical record. This will help clinicians make a more informed decision about the diagnosis. It is important that users give us feedback on the use of the software so that we can improve the model. This is important because it helps us to better understand how the software is being used and how we can improve it.
We are currently working on developing new features for the software. We are looking for collaborators who want to investigate the use of these models in diagnostics. If you are interested, please contact us for more information.
In a recent preliminary evaluation conducted by Foundation 29 using 200 synthetic patient cases generated by language models, DxGPT demonstrated some ability to suggest correct diagnoses of rare diseases. Specifically, when evaluated for strict accuracy (i.e., whether the correct diagnosis appeared in first position), DxGPT obtained an accuracy of 67.5%, getting the diagnosis right in 135 of the 200 cases. If a more flexible metric is considered, counting correct diagnoses in both the first and first 5 positions, the accuracy increases to 88.5% (177 out of 200 cases). While these initial results are encouraging, DxGPT is still at an early stage of development.
For more details on the evaluation methodology and results, please see our GitHub repository and the associated article.
The server code is here: DxGPT SERVER
