ac2art
is a Python package for the supervised learning of the
acoustic-to-articulatory inversion task using neural networks. It integrates a
data processing pipeline that already interfaces some reference corpora and
is ready to be expanded to additional ones, as well as a neural networks API
to build, train and save models of various and modular architectures, that
relies in the background on the low-level API of Tensorflow
.
Beside the systematic documentation of all implemented functions in the form
of docstrings, basic knowledge of how ac2art
should be used and is structured
is provided in the form of a hands-on tutorial (ipython notebook with an html
counterpart) and three markdown files covering respectively the ac2art.corpora
module, the ac2art.networks
one and the ac2art.networks.NeuralNetwork
API.
Those files are to be found in the docs
folder of this repository.
Python
A Python 3 (>= 3.4) installation is required to run ac2art
.
Additionally, a Python 2.7 installation is required as some key functionalities rely on interfaced third-party Python 2 packages (see below).
Python 2 third-party packages
The (optional) use of kaldi to compute acoustic
features depends on the abkhazia
package. The computation of ABX discriminability metrics depends on
ABXpy. Both those packages should be
installed manually before installing ac2art
(and will be checked for at
installation time). Please do not use pip
to install those packages,
are they are not maintained up-to-date on Pypy and some key functionalities
would therefore be missing and cause crashes.
Ptyhon 3 third-party packages
ac2art
also depends on the following third-party Python 3 packages, which
will be automately installed as part of the installation procedure :
h5features
, numpy
, pandas
, scipy
, sphfile
and tensorflow
.
You may want to manually compile and install the latter depending on your
system. Note that as of now, ac2art
does not take advantage of GPUs.
To this day, ac2art
supports the following data corpora :
MNGU0,
MOCHA-TIMIT and
MSPKA (with some to-be-fixed flaws for the latter).
One may choose not to download them prior to installing the package ; however,
doing so enables the inclusion of their path to the config.json
file (see
following section).
So as to work with the data from the corpora, please do not rename nor
restructure the downloaded files and folders, and simply copy them to any
location that you see fit (and will then have to fill in the config.json
file).
For the MNGU0 and MOCHA-TIMIT packages, acoustic features computation using
Kaldi requires the use of completed phoneme labeling files, which were manually
filled based on those distributed with each corpus and are yet to be released.
In the meanwhile, one may use an ac2art
built-in feature to compute MFCC,
resulting in slightly different coefficients and not including the pitch
features which Kaldi adds.
ac2art
depends on a config.json
file that should be created and set up
before running its installation, which will be copied somewhere on your system
during the installation. It may be updated later using the built-in
ac2art.utils.update_config
function, which may however require you to run
python with root rights depending on the way you installed the package.
An example_config.json
file is provided as part of this repository, which
can be renamed to config.json
and filled out, as it contains the fields that
you should want to use. Those include the mandatory paths to your kaldi and
ABXpy installations, as well as the (optional but fairly crucial) paths to the
folders containing the raw data from the supported corpora, and paths to
folders where to store the processed versions of those datasets (including
acoustic and articulatory features in both raw and normalized versions, as
well as produced ABXpy h5 features files and output scores).
To install ac2art
on your machine:
- Download a copy of the Git repository:
git clone https://github.com/pandrey-fr/ac2art.git
- Set up the
config.json
file in the downloaded folder (see previous section). - In the command line,
cd
to the folder, then runpython3 setup.py install
(either with root rights, or adding the--user
option for local installation).
Copyright 2018 Paul Andrey
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
If you run into any issue regarding ac2art
, you may open an issue on this
Github repository, or contact the author.