Skip to content

Commit

Permalink
Added phrase-based test case. Phrase-based decoder (#172) functioning.
Browse files Browse the repository at this point in the history
  • Loading branch information
mjpost committed Oct 22, 2014
1 parent b94f2aa commit 105316f
Show file tree
Hide file tree
Showing 6 changed files with 46 additions and 0 deletions.
27 changes: 27 additions & 0 deletions test/decoder/phrase/config
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
tm = phrase pt 0 rules.1.gz
lm = kenlm 5 true false 100 lm.1.gz

mark-oovs = false
pop-limit = 10
top-n = 1

output-format = %i ||| %s ||| %f ||| %c

include-align-index = true
reordering-limit = 6

# And these are the feature functions to activate.
feature-function = OOVPenalty
feature-function = WordPenalty
feature-function = Distortion
feature-function = PhrasePenalty -owner pt

OOVPenalty -100
Distortion 0.114849
WordPenalty -0.201544
PhrasePenalty -0.236965
tm_pt_0 0.0370068
tm_pt_1 0.0495759
tm_pt_2 0.196742
tm_pt_3 0.0745423
lm_0 0.204412452147565
1 change: 1 addition & 0 deletions test/decoder/phrase/corpus.es
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
una estrategia republicana para obstaculizar la reelección de Obama
Binary file added test/decoder/phrase/lm.1.gz
Binary file not shown.
1 change: 1 addition & 0 deletions test/decoder/phrase/output.gold
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0 ||| a strategy |0-1| republican |2-2| to hinder |3-4| reelection |5-6| Obama |7-8| ||| Distortion=0.000 WordPenalty=-3.040 PhrasePenalty=5.000 tm_pt_0=-9.702 tm_pt_1=-10.800 tm_pt_2=-7.543 tm_pt_3=-8.555 lm_0=-19.117 ||| -7.496
Binary file added test/decoder/phrase/rules.1.gz
Binary file not shown.
17 changes: 17 additions & 0 deletions test/decoder/phrase/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash

set -u

cat corpus.es | $JOSHUA/bin/joshua-decoder -threads 1 -c config > output 2> log

# Compare
diff -u output output.gold > diff

if [ $? -eq 0 ]; then
rm -f diff output log
exit 0
else
exit 1
fi


0 comments on commit 105316f

Please sign in to comment.