README.md

CNTK on OpenPAI

This guide introduces how to run CNTK job on OpenPAI. The following contents show some basic CNTK examples, other customized CNTK code can be run similarly.

CNTK grapheme-to-phoneme examples

To run CNTK examples in OpenPAI, you need to prepare a job configuration file and submit it through webportal.

OpenPAI packaged the docker env required by the job for user to use. User could refer to DOCKER.md to customize this example docker env. If user have built a customized image and pushed it to Docker Hub, replace our pre-built image openpai/pai.example.caffe with your own.

Here're some configuration file examples:

grapheme-to-phoneme

{
  "jobName": "cntk-g2p",
  "image": "openpai/pai.example.cntk",

  // prepare cmudict corpus in CNTK format https://git.io/vbT5A and upload to hdfs
  "dataDir": "$PAI_DEFAULT_FS_URI/path/data",
  // make a new dir for output on hdfs
  "outputDir": "$PAI_DEFAULT_FS_URI/path/output",
  // prepare g2p training script cntk-example.sh and upload to hdfs
  "codeDir": "$PAI_DEFAULT_FS_URI/path/code",

  "taskRoles": [
    {
      "name": "g2p_train",
      "taskNumber": 1,
      "cpuNumber": 8,
      "memoryMB": 16384,
      "gpuNumber": 1,
      "command": "cd code && /bin/bash cntk-g2p.sh"
    }
  ]
}

For more details on how to write a job configuration file, please refer to job tutorial.