Developing tgclients
Here are notes how to setup an development environment to develop the tgclients library itself.
Setup development environment
Prerequisites
Python > 3.7
Create/activate virtual environment (ensure you use the correct python version!).
python -m venv venv . venv/bin/activate pip install --upgrade pip
Install requirements.
pip install -e .[dev]
Generate a requirements.txt
If a requirements.txt is needed, it can be generated out of setup.py with pip-tools:
pip-compile setup.cfg
If a requirements.txt for the dev dependencies (or a requirements.dev.txt) is needed, enter:
pip-compile setup.cfg --extra dev --allow-unsafe -o requirements.dev.txt
ICU dependency
If you rely on filename_from_metadata() feature you should possibly install PyICU, as this makes sure the same transliteration as in TextGrid aggregator and TextGridLab is used. Then install with:
pip install -e .[icu,dev]
or just
pip install -e .[icu]
There is a minimal fallback implememted for use without PyICU, which is only sufficient to pass the integration tests.
Contributing
Commit convention:
Style constraints:
Code: PEP 8
Documentation: Google style docstrings (Chapter 3.8 from Google styleguide)
Coding constraints:
Objects that are not supposed to be used outside the current scope MUST be named starting with
_
(underscore): PEP 316
For your convenience, pre-commit hooks are configured to check against these constraints. Provided, you have installed the development requirements (see above), activate pre-commit
to run on every git commit
:
pre-commit install
Also, a helper with conventional commits is installed with the development requirements that you could leverage to easily comply with it. Just use cz c
instead of git commit
Testing
Unit Tests
pytest
Integration Tests
For Integration tests create a project in TextGridLab and get a SessionID.
Then create a file ‘.env’ containing the following entries:
SESSION_ID=YOUR-SESSION-ID
PROJECT_ID=YOUR-PROJECT-ID
PROJECT2_ID=ANOTHER-PROJECT-ID
For testing on another TextGrid server than the default production system you may also set the TEXTGRID_HOST
environment variable.
set -o allexport; source .env; set +o allexport
pytest --integration
to capture print() output (only works if assert failed):
pytest -o log_cli=true --capture=sys --integration
to capture with debug log enabled
pytest -o log_cli=true --log-cli-level=DEBUG --capture=sys --integration
Logging
The tgclients log communicaction problems with the services with log level WARNING, to have them visible in Jupyter notebooks. If you use the clients and do not want to pollute your log files you may change the log level of the clients to ERROR, e.g.:
import logging
logging.getLogger('tgclients').setLevel(logging.ERROR)
or more specific, e.g. for not getting crud warnings:
import logging
logging.getLogger('tgclients.crud').setLevel(logging.ERROR)
API doc generation
generated with:
cd docs
rm -r apidoc
sphinx-apidoc ../src/ -o apidoc
Databinding
to re-generate the databinding do
pip install xsdata[cli]
cd src
xsdata https://gitlab.gwdg.de/dariah-de/textgridrep/tg-search/-/raw/main/tgsearch-api/src/main/resources/tgsearch.xsd --package tgclients.databinding --docstring-style Google
tgpublish databinding from wadl: note(2024-06-14): seems like the wadl is not generated anymore with newer cxf versions. so we saved the old xsd in this git repo in ./src/tgpub.xsd
cd src
# download wadl and extract the xsd schema part with xpath
curl https://textgridlab.org/1.0/tgpublish/?_wadl | xmllint --xpath "//*[local-name()='schema']" - > tgpub.xsd
# generate databinding
xsdata tgpub.xsd --package tgclients.databinding.tgpublish --docstring-style Google