PyTorch functions to compute meaningful probabilistic relevance scores from PHOC (Pyramid of Histograms of Characters) embeddings. Although they are called Pyramid of Histograms of Characters, in practice they are a Pyramid of Bag of Characters. At the end, each word is represented by a high-dimensional binary vector.
See the wiki for additional details.
The library provides two functions: cphoc and pphoc, which are
similar to SciPy's cdist and pdist:
Both functions can operate with PHOC embeddings in the probability space (where each dimension is a real number in the range [0, 1]), or in the log-probability space (where each dimension is the logarithm of a probability). These are also sometimes refered to as the Real and Log semirings.
import torch
from prob_phoc import cphoc, pphoc
x = torch.Tensor(...)
y = torch.Tensor(...)
# Compute the log-relevance scores between all pairs of rows in x, y.
# Note: x and y must have the PHOC log-probabilities.
logprob = cphoc(x, y)
# This is equivalent to:
logprob = cphoc(x, y, method="sum_prod_log")
# If your matrices have probabilities instead of log-probabilities, use:
prob = cphoc(x, y, method="sum_prob_real")
# Compute the log-relevance scores between all pairs of distinct rows in x.
# Note: The output is a vector with N * (N - 1) / 2 elements.
logprob = pphoc(x)The easiest way is to install the package from PyPI:
pip install prob-phocIf you want to install the latest version from the repository, clone it and use the setup.py script to compile and install the library.
python setup.py installYou will need a C++11 compiler (tested with GCC 4.9). If you want to compile with CUDA support, you will also need to install the CUDA Toolkit (tested with versions 8.0, 9.0 and 10.0)
After the installation, you can run the tests to ensure that everything is working fine.
python -m prob_phoc.testI have also some benchmarks to compare CPU vs. CUDA, for different matrix sizes and float precision. These take quite a long to run, so don't hold your breath.
python -m prob_phoc.benchmark