Skip to content

feat: Add pipeline_kwargs to TransformersSummarizer#6628

Merged
masci merged 6 commits into
v1.xfrom
summarizer-speed
Jan 8, 2024
Merged

feat: Add pipeline_kwargs to TransformersSummarizer#6628
masci merged 6 commits into
v1.xfrom
summarizer-speed

Conversation

@sjrl
Copy link
Copy Markdown
Contributor

@sjrl sjrl commented Dec 22, 2023

Related Issues

Proposed Changes:

  • fixes slow inference and indexing times for a client's pipeline that uses the TransformerSummarizer
  • use batching in the predict method since multiple documents are usually passed at inference time
  • allow the model to be loaded in torch.float16 by adding pipeline_kwargs to the init method

How did you test it?

  • added unit tests

Notes for the reviewer

Checklist

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Dec 22, 2023
@sjrl sjrl added the 1.x label Dec 22, 2023
@sjrl sjrl marked this pull request as ready for review January 2, 2024 08:47
@sjrl sjrl requested review from a team as code owners January 2, 2024 08:47
@sjrl sjrl requested review from dfokina and masci and removed request for a team January 2, 2024 08:47
Copy link
Copy Markdown
Contributor

@masci masci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@masci masci merged commit 7949993 into v1.x Jan 8, 2024
@masci masci deleted the summarizer-speed branch January 8, 2024 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants