Skip to content
This repository was archived by the owner on Dec 29, 2022. It is now read-only.
This repository was archived by the owner on Dec 29, 2022. It is now read-only.

Deprecate non-standard BLEU scripts #365

@kpu

Description

@kpu

multi-bleu.perl has been deprecated for years now because it encourages people to use non-standard tokenization. This repository contains another non-standard BLEU implementation that a user might not notice they are using: https://github.com/google/seq2seq/blob/7f485894d412e8d81ce0e07977831865e44309ce/seq2seq/metrics/bleu.py

A paper by @mjpost https://www.aclweb.org/anthology/W18-6319/ shows how much this can vary.

Please put fat warnings on non-standard BLEU scripts that they are not appropriate for publication, as Moses has. And remove multi-bleu.perl from examples.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions