This release adds a new optional feature for data ingestion that sends each figure in a document to Azure Content Understanding to generate a description. This is an alternative approach to our vision approach for handling documents with images, though it could even be used in conjunction with it.
For example, for this document:
The graph is replaced by this description:
The description is stored as HTML in the chunk, for maximal expressivity:
<figure>
<figcaption>Impact of Interest Rates, Inflation, and GDP Growth on Financial Markets<br>
The line graph titled 'On Financial Markets' displays trends in Interest Rates, Inflation Data, and GDP Growth from 2018 to 2023. The x-axis represents the years, while the y-axis shows percentage values ranging from -5 to 5. Interest Rates start at 2% in 2018, dip to 1% in 2020, and rise to 2% by 2023. Inflation Data begins at 2%, peaks at 3% in 2022, and slightly decreases to 2.5% in 2023. GDP Growth shows a significant drop to -4% in 2020, recovers to 3% in 2021, and stabilizes at 2% by 2023.
<table>
<thead>
<tr>
<td>Year</td>
<td>Interest Rates %</td>
<td>Inflation Data %</td>
<td>GDP Growth %</td>
</tr>
</thead>
<tbody>
<tr>
<td>2018</td>
<td>2</td>
<td>2</td>
<td>2</td>
</tr>
<tr>
<td>2019</td>
<td>2</td>
<td>2.5</td>
<td>1</td>
</tr>
<tr>
<td>2020</td>
<td>1</td>
<td>1.5</td>
<td>-4</td>
</tr>
<tr>
<td>2021</td>
<td>1.5</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>2022</td>
<td>2</td>
<td>3</td>
<td>2</td>
</tr>
<tr>
<td>2023</td>
<td>2</td>
<td>2.</td>
</tr>
</tbody>
</table></figure>
See the doc for enabling the feature:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-media-description-with-azure-content-understanding
Please try it out and give feedback.
What's Changed
- Add media description feature using Azure Content Understanding by @pamelafox in #2195
Full Changelog: 2024-12-02...2024-12-09
Contributors
pamelafox