Documentation

[dirkroorda]

The Text-Fabric docs are growing into a bewildering amount of information. We need a kind of documentation that is written around a clear path from beginner-level to the essentials of Text-Fabric.

Here are a few ideas.

Split the Essentials Guide in two parts:
(A) for those who do not know Python
(B) for those who have beginner-level Python skills

[dirkroorda]

Scenario: Researcher (non-programming)

Before installing anything: full-text interface

Navigate to the full-text interface in your browser:
https://annotation.github.io/app-dhammapada

Learn how to full-text search the corpus.
Explain the levels and layers.
Introduce node types, nodes and features.

Navigate to other corpora with their layered-search interfaces.
nena, fusus, bhsa

Install a single app: JupyterLab-Desktop

get JupyterLab desktop (it installs as a single app, it has its own python)

learn to install text-fabric within this jupyter lab (%pip install text-fabric)

start the text-fabric browser from this jupyter lab (!text-fabric dhammapada)

learn the capabilities of the text-fabric browser

• explore the links to the corpus documentation and to the Text-Fabric documentation.

• especially study the feature documentation of your corpus

• call up nodes with the node pad

• call up sections with the sections pad

• perform queries with the search pad

See results in context.
Click on the links to external websites with the same portion of text
Expand the underlying data.
Play with the options for the representation of this data

Now enhace your query skills.
Walk through the search usage document and try things out.

Try other corpora as well: sblgnt, dss, bhsa, fusus, missieven

• get a feel for what is in text-fabric and what is in the corpus

Saving and exporting your work

Save your work in the browser. Organize them in jobs.
Export jobs and import them again.
Export the results of a query and open them in Excel.

Publishing your work

Store all data of a job as a data set in a repository:

• Zenodo or
• GitHub (with Software Heritage Archive) or
• DANS

and obtain a DOI (or SHA identifier of the Software Heritage Archive).
Write a mini-article (< 1 page) about your dataset and cite your dataset by means of the DOI.
Add the mini-article to your profile on academia.edu, or your home institution, or Linked-In, or GitHub, or whatever public place.

[dirkroorda]

Scenario Researcher (programming, analysis only)

This continues from where the non-programming scenario stops.

We still work in the JupyterLab-Desktop environment.

Additional install options

Install other python modules, e.g. python-Levenshtein
Let JupyterLab-Desktop work with the Python that you already have on your system

Install jupyterlab in the Python you already have on your system.

Repository based

Familiarize yourself with GitHub.
Create/use an org on Github where you can develop your work, and create a repo in it.
Take care of the license, and the .gitignore file.
Use the README.

On your computer, in your home directory, make a github directory.
In there, create a directory with the same name of your org in the online github. Go to that directory, and clone your new repo there.

Compute

Make a directory notebooks in there, and create a new notebook there, from within your JupyterLab environment.

Load a corpus by means of the incantation, like
A = use("dhammapada").

Now we have to describe a sequence of steps by which you understand

• getting information about the corpus C.levels.data, C.characters.data
• walking through nodes N.walk()
• retrieving feature values F.pali.v(n)
• retrieving edge feature values E.oslots.s(n)
• going up and down L.d(n), L.u(n)
• printing text T.text(n)

Read the in-depth doc about the data model

Search

We revisit what you did in the Text-Fabric browser.
Do your searches in the notebook now.

• understand results tuples
• show them with A.table() and A.show()
• play with display options (e.g. condense, condenseType, baseTypes)
• use multiple highlight colors
• display additional features

Search and computation

We give some exercises to

• check search results by means of hand-coding
• combine search and hand-coding in cases where search templates are not expressive enough to pin-point what you have in mind.

[dirkroorda]

Scenario Researcher (programming, feature creation)

This scenario follows on the previous one.
We use the same setting, JupyterLab and GitHub.
We use programming to analyse a corpus and to add features to it as the result of analysis.

We learn how to publish such features in data modules that others can import.
We learn how to import data modules from others alongside our own modules.
We learn how to use all those modules together in queries and hand coding that use them all.

We learn how to take care of long-term preservation, by making releases on Github and transport releases to Zenodo and the Software Heritage Archive.

[dirkroorda]

Scenario Corpus developer

How to convert existing corpora into text-fabric (walker conversion)
Make tutorials for your corpus.
Make feature documentation.
Link to existing online incarnations of your corpus.
How to create a tf-app for your corpus.
How to set up layered search interfaces for your corpus.
How to preserve your corpus.

[dirkroorda]

Scenario Corpus Developer advanced

How to use volume support.

How to create node mappings between versions of the corpus.

How to combine and modify datasets.