Add LLM query example with Ollama

[ehsantn]

Changes included in this PR

Adds an example that demonstrates preprocessing data and querying an LLM using Bodo (with Ollama).

Testing strategy

Tested it locally.

User facing changes

A new example.

Checklist

• [N/A] Pipelines passed before requesting review. To run CI you must include [run CI] in your commit message.
• I am familiar with the Contributing Guide
• I have installed + ran pre-commit hooks.

[scott-routledge2]

LGTM, is there a larger dataset of queries we could use to show performance benefits over python?

[ehsantn]

Running LLMs locally without GPUs is very slow so larger data would make it not usable.

[marquisdepolis]

Ollama is meant for local runs, smaller scale w/o GPUs. Next we can use OpenAI compatible APIs, via https://github.com/andrewyng/aisuite or LiteLLM, at larger scale, and see how that helps.

…

On Mon, Dec 30, 2024 at 10:03 AM Ehsan Totoni ***@***.***> wrote: Running LLMs locally without GPUs is very slow so larger data would make it not usable. — Reply to this email directly, view it on GitHub <#94 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE7INWKHOM6WHXL3B3WDRHL2IGDF7AVCNFSM6AAAAABUJE3ALSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRVG43TENJWGI> . You are receiving this because your review was requested.Message ID: ***@***.***>

[marquisdepolis]

I'd edit to model="llama3", or even make that an outside variable, to keep it cleaner. And add a comment next to raw_prompts*10 that this just runs the examples 10x times.

[ehsantn]

Done.