spotterbase.evaluate package

Submodules

spotterbase.evaluate.annocollection module

class spotterbase.evaluate.annocollection.AnnoCollection(fragment_annos_by_source: 'dict[Uri, list[AnnoWithFragTarget]]' = <factory>)

Bases: object

fragment_annos_by_source: dict[Uri, list[AnnoWithFragTarget]]

classmethod from_records(records: list[Record]) → AnnoCollection

class spotterbase.evaluate.annocollection.AnnoWithFragTarget(anno: 'Annotation', target: 'FragmentTarget')

Bases: object

anno: Annotation

target: FragmentTarget

spotterbase.evaluate.rangecmp module

class spotterbase.evaluate.rangecmp.AnnoWithFragTargetPair(golden: 'AnnoWithFragTarget', prediction: 'AnnoWithFragTarget')

Bases: object

get_doc_uri() → Uri

golden: AnnoWithFragTarget

prediction: AnnoWithFragTarget

class spotterbase.evaluate.rangecmp.OffsetEquiConfig(invisible_nodes: set[str], visible_tags: set[str])

Bases: object

invisible_nodes:

nodes that can be completely ignored (i.e. it doesn’t make a difference if they are included or not).

visible_tags:

by default, tags are invisible. So if you have ab<span>def</span>gh, it does not make a difference if you annotate <span>def</span> or just def or just <span>def. By adding span to visible_tags, you can make the span visible. This is relevant for meaning-carrying tags (e.g. <msqrt> in MathML).

Clarification:

‘x’ in invisible_nodes is stronger than ‘x’ not in visible_tags, because it means that the content of x is completely irrelevant, while the content of a tag not in visible_tags is still relevant, just whether the “open” and “close” tags are included is irrelevant.

Note: This is a prototype implementation. In the future, we need support more complex cases

(e.g. tags that are only visible if they contain certain attributes).

invisible_nodes: set[str]

visible_tags: set[str]

class spotterbase.evaluate.rangecmp.OffsetEquis(invis_ranges: Iterable[tuple[int, int]], doc: Document)

Bases: object

Essentially an equivalence relation on offsets

classmethod from_doc_simple(doc: Document, offset_equi_config: OffsetEquiConfig) → OffsetEquis

minimize_range(dor: DomOffsetRange) → DomOffsetRange

Returns the shortest range that is equivalent to the input range

class spotterbase.evaluate.rangecmp.RangeMatching(precise_matches: 'list[AnnoWithFragTargetPair]', overlaps: 'list[AnnoWithFragTargetPair]', golden_only: 'list[AnnoWithFragTarget]', prediction_only: 'list[AnnoWithFragTarget]', golden_in_multimatches: 'list[AnnoWithFragTarget]', prediction_in_multimatches: 'list[AnnoWithFragTarget]', multimatches: 'list[AnnoWithFragTargetPair]')

Bases: object

classmethod from_anno_collections(golden: AnnoCollection, prediction: AnnoCollection, offset_equi_config: OffsetEquiConfig) → RangeMatching

golden_in_multimatches: list[AnnoWithFragTarget]

golden_only: list[AnnoWithFragTarget]

multimatches: list[AnnoWithFragTargetPair]

overlaps: list[AnnoWithFragTargetPair]

precise_matches: list[AnnoWithFragTargetPair]

prediction_in_multimatches: list[AnnoWithFragTarget]

prediction_only: list[AnnoWithFragTarget]

print_overlap_details()

Prints overlapping annotations in detail to help find out why they only overlap instead of being equal

Module contents