mirror of
https://github.com/nhammer514/textfiles-politics.git
synced 2025-05-07 09:05:12 -04:00
work on Python with Jermaine
This commit is contained in:
parent
1f771f237a
commit
01b77812fe
691 changed files with 305465 additions and 61600 deletions
38
pythonCode/markdownExample.md
Normal file
38
pythonCode/markdownExample.md
Normal file
|
@ -0,0 +1,38 @@
|
|||
# My Documentation
|
||||
|
||||
Okay, so we are working with this Python script that does natural language processing.
|
||||
|
||||
* Python File 1 is the [PersonTagger](personTagger.py)
|
||||
* Python File 2 is the [CultureTagger](cultureTagger.py)
|
||||
|
||||
This is a code block from your project Python that we were just working on:
|
||||
```py
|
||||
config = {"spans_key": None, "annotate_ents": True, "overwrite": True, "validate": True}
|
||||
ruler = nlp.add_pipe("span_ruler", before="ner", config=config)
|
||||
# 2023-04-07: ebb: NOTE: before="ner" setting seems to allow the spaCy NER rules to prevail over these patterns where
|
||||
# there is a conflict.
|
||||
# after="ner" means that the spaCy ner is TOTALLY OVERWRITTEN and invalidated by our patterns.
|
||||
|
||||
# Notes: Mattingly has this: ruler = nlp.add_pipe("entity_ruler", after="ner", config={"validate": True})
|
||||
# But this only works when spaCy doesn't recognize a word / phrase as a named entity of any kind.
|
||||
# If it recognizes a named entity but tags it wrong, we correct it with the span_ruler, not the entity_ruler
|
||||
patterns = [
|
||||
{"label": "NULL", "pattern": [{"TEXT" : {"REGEX": "^-\w+?"}}]},
|
||||
{"label": "NULL", "pattern": [{"TEXT" : {"REGEX": "^\w$"}}]},
|
||||
{"label": "GPE", "pattern": [{"TEXT" : {"REGEX": "Babylon(ia)?"}}]},
|
||||
{"label": "NULL", "pattern": "di"},
|
||||
{"label": "ORG", "pattern": "Falangist"},
|
||||
{"label": "NORP", "pattern": "Dropa"},
|
||||
{"label": "GPE", "pattern": "Nazareth"},
|
||||
{"label": "NULL", "pattern": "Bab"},
|
||||
]
|
||||
ruler.add_patterns(patterns)
|
||||
```
|
||||
|
||||
Now, here is what is happening in this code.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
Loading…
Add table
Add a link
Reference in a new issue