syndisco.util package¶

Submodules¶

syndisco.util.file_util module¶

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

You may contact the author at tsirbasdim@gmail.com

syndisco.util.file_util.dict_to_json(dictionary: dict[str, Any], output_path: str | Path) → None¶

Write a python dictionary to a file in JSON format.

Parameters:

dictionary (dict[str, Any]) – The dictionary to be serialized
output_path (str | Path) – The output path to the file. If the parent directories do not exist, they will be created.

syndisco.util.file_util.ensure_parent_directories_exist(output_path: str | Path) → None¶: Create all parent directories if they do not exist. :param output_path: the path for which parent dirs will be generated :type output_path: str | Path

syndisco.util.file_util.generate_datetime_filename(output_dir: str | Path | None = None, timestamp_format: str = '%y-%m-%d-%H-%M', file_ending: str = '') → Path¶

Generate a filename based on the current date and time.

Parameters:

output_dir (str | Path, optional) – The path to the generated file, defaults to None
timestamp_format (str, optional) – strftime format, defaults to “%y-%m-%d-%H-%M”
file_ending (str) – The ending of the file (e.g ‘.json’)

Returns:

the full path for the generated file

Return type:

str

syndisco.util.file_util.read_file(path: str | Path) → str¶

Return the contents of a file

Parameters:: path (str | Path) – the path of the file
Returns:: the file’s contents
Return type:: str

syndisco.util.file_util.read_files_from_directory(directory: str | Path) → list[str]¶

Reads all files from a given directory.

Parameters:: directory (str | Path) – the root directory from which to load files (NOT recursively!)
Raises:: ValueError – if the directory does not exist
Returns:: Returns a list of parsed file content.
Return type:: list[str | dict]

syndisco.util.file_util.read_json_file(path: str | Path) → dict[str, Any]¶: Read a JSON file :param path: the path of the file :type path: str | Path :return: the file’s contents :rtype: dict[str, Any]

syndisco.util.logging_util module¶

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

You may contact the author at tsirbasdim@gmail.com

syndisco.util.logging_util.logging_setup(print_to_terminal: bool, write_to_file: bool, logs_dir: str | Path | None = None, level: str = 'debug', use_colors: bool = True, log_warnings: bool = True) → None¶

Create the logger configuration.

Parameters:

print_to_terminal (bool) – whether to print logs to the screen
write_to_file (bool) – whether to write logs to a file. Needs logs_dir to be specified.
logs_dir (Optional[str | Path], optional) – the directory where the logs will be placed, defaults to None
level – the logging level, defaults to logging.DEBUG
use_colors (bool, defaults to True) – whether to color the output.
log_warnings (bool, defaults to True) – whether to log library warnings

syndisco.util.logging_util.timing(f: Callable) → Any¶

Decorator which logs the execution time of a function.

Parameters:: f (Function) – the function to be timed
Returns:: the result of the function
Return type:: _type_

syndisco.util.model_util module¶

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

You may contact the author at tsirbasdim@gmail.com

class syndisco.util.model_util.ModelManager(model_path: str, max_new_tokens: int, disallowed_strings: list[str], model_pseudoname: str | None = None)¶

Bases: object

A Singleton class initializing and managing access to a single, unique instance of a model.

get() → BaseModel¶

Get a reference to the protected model instance. First invocation loads the instance to runtime.

Raises:: NotImplementedError – if an incompatible library_type is given in the yaml_data of the constructor
Returns:: The initialized model instance.
Return type:: model.Model

syndisco.util package¶

Submodules¶

syndisco.util.file_util module¶

syndisco.util.logging_util module¶

syndisco.util.model_util module¶

Module contents¶

Table of Contents

This Page