Skip to content

examples: add Fetch And Embed Data Job Example#3065

Merged
yonitoo merged 11 commits into
mainfrom
topic/ysalambashev/add-example-embedding-job
Feb 1, 2024
Merged

examples: add Fetch And Embed Data Job Example#3065
yonitoo merged 11 commits into
mainfrom
topic/ysalambashev/add-example-embedding-job

Conversation

@yonitoo

@yonitoo yonitoo commented Jan 30, 2024

Copy link
Copy Markdown
Contributor

What:
Add Data Job that fetches public Confluence data and then creates embeddings out of it.

Why:
To have a job that creates embeddings out of Confluence documents.

Closes #2992

Signed-off-by: Yoan Salambashev <yoan.salambashev@broadcom.com>
Signed-off by: Yoan Salambashev <yoan.salambashev@broadcom.com>

@antoniivanov antoniivanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.

Comment thread examples/fetch-embed-job-example/10_fetch_confluence_space.py Outdated
Comment thread examples/fetch-embed-job-example/20_clean_and_embed_data.py
Comment thread examples/fetch-embed-job-example/20_clean_and_embed_data.py Outdated
Comment thread examples/fetch-embed-job-example/10_fetch_confluence_space.py Outdated
Signed-off by: Yoan Salambashev <yoan.salambashev@broadcom.com>
Signed-off by: Yoan Salambashev <yoan.salambashev@broadcom.com>
Signed-off by: Yoan Salambashev <yoan.salambashev@broadcom.com>
Signed-off by: Yoan Salambashev <yoan.salambashev@broadcom.com>
Signed-off by: Yoan Salambashev <yoan.salambashev@broadcom.com>
Comment thread examples/fetch-embed-job-example/requirements.txt
Comment thread examples/fetch-embed-job-example/20_clean_and_embed_data.py
@yonitoo
yonitoo requested a review from duyguHsnHsn February 1, 2024 11:25
@yonitoo
yonitoo merged commit 09c49ba into main Feb 1, 2024
@yonitoo
yonitoo deleted the topic/ysalambashev/add-example-embedding-job branch February 1, 2024 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decide on which API should be used for embedding chunks of text

4 participants