Skip to content

Refactor/add configurable parameters#866

Merged
alytarik merged 17 commits intorelease/1.8.0from
refactor/add-configurable-parameters
Nov 3, 2023
Merged

Refactor/add configurable parameters#866
alytarik merged 17 commits intorelease/1.8.0from
refactor/add-configurable-parameters

Conversation

@alytarik
Copy link
Copy Markdown
Contributor

@alytarik alytarik commented Oct 30, 2023 •
edited
Loading

Description

This PR adds configurable parameters instead of some hardcoded values in

  • CrowS-Pairs
    • diff_treshold
    • filter_treshold
  • Wino-Bias
    • diff_treshold
  • Clinical, Disinformation, Factuality,Translation
    • sentence_transformer

➤ Fixes #857

Type of change

  • New feature (non-breaking change which adds functionality)

Usage

tests:
  wino-bias:
    gender-occupational-stereotype:
      min_pass_rate: 0.70
      diff_threshold: 0.03
tests:
  crows-pairs:
    common-stereotypes:
      min_pass_rate: 0.70
      diff_threshold: 0.10
      filter_threshold: 0.15

Checklist:

  • I've added Google style docstrings to my code.
  • I've used pydantic for typing when/where necessary.
  • I have linted my code
  • I have added tests to cover my changes.

Screenshots (if appropriate):

@alytarik alytarik self-assigned this Oct 30, 2023
@alytarik alytarik linked an issue Oct 30, 2023 that may be closed by this pull request
Make parameters configurable #857
Closed
@alytarik alytarik marked this pull request as ready for review October 30, 2023 08:10
@alytarik alytarik added 💡Enhancements Something can be improved v2.1.0 Issue or request to be done in v2.1.0 release labels Oct 30, 2023
chakravarthik27
chakravarthik27 reviewed Oct 30, 2023
View reviewed changes
langtest/transform/__init__.py Outdated
Empty list

"""
print(self.tests)
Copy link
Copy Markdown
Collaborator

@chakravarthik27 chakravarthik27 Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we need this print()

Copy link
Copy Markdown
Contributor Author

@alytarik alytarik Oct 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testing purposes, removed now

@alytarik alytarik changed the base branch from refactor/delete-dead-code to release/1.8.0 October 30, 2023 12:17
@alytarik alytarik changed the base branch from release/1.8.0 to refactor/delete-dead-code October 30, 2023 12:27
@alytarik alytarik changed the base branch from refactor/delete-dead-code to release/1.8.0 November 3, 2023 10:00
@alytarik alytarik merged commit 3ce94b8 into release/1.8.0 Nov 3, 2023
@ArshaanNazir ArshaanNazir deleted the refactor/add-configurable-parameters branch November 16, 2023 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chakravarthik27 chakravarthik27 chakravarthik27 approved these changes

Assignees

@alytarik alytarik

Labels

💡Enhancements Something can be improved v2.1.0 Issue or request to be done in v2.1.0 release

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Make parameters configurable

2 participants

@alytarik @chakravarthik27