forked from apache/pig
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES.txt
375 lines (224 loc) · 12.3 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
Pig Change Log
Trunk (unreleased changes)
INCOMPATIBLE CHANGES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
Release 0.1.1 - Unreleased
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
PIG-253: integration with hadoop-18
BUG FIXES
PIG-342: Fix DistinctDataBag to recalculate size after it has spilled. (bdimcheff via gates)
Release 0.1.0 - - 2008-09-11
INCOMPATIBLE CHANGES
PIG-123: requires escape of '\' in chars and string
NEW FEATURES
PIG-20 Added custom comparator functions for order by (phunt via gates)
PIG-94: Streaming implementation
PIG-58: parameter substitution
PIG-55: added custom splitter (groves via olgan)
PIG-59: Add a new ILLUSTRATE command (shubhamc via gates).
PIG-256: Added variable argument support for UDFs (pi_song)
IMPROVEMENTS
PIG-8 added binary comparator (olgan)
PIG-11 Add capability to search for jar file to register. (antmagna via
olgan)
PIG-7: Added use of combiner in some restricted cases. (gates)
PIG-47: Added methods to DataMap to provide access to its content
PIG-30: Rewrote DataBags to better handle decisions of when to spill to
disk and to spill more intelligently. (gates)
PIG-12: Added time stamps to log4j messages (phunt via gates).
PIG-44: Added adaptive decision of the number of records to hold in memory
before spilling (utkarsh)
PIG-56: Made DataBag implement Iterable. (groves via gates)
PIG-39: created more efficient version of read (spullara via olgan)
PIG-32: ABstraction layer (olgan)
PIG-83: Change everything except grunt and Main (PigServer on down) to use
common logging abstraction instead of log4j. By default in grunt, log4j
still used as logging layer. Also converted all System.out/err.println
statements to use logging instead. (francisoud via gates)
PIG-13: adding version to the system (joa23 via olgan)
PIG-113: Make explain output more understandable (pi_song via gates)
PIG-120: Support map reduce in local mode. To do this user needs to
specify execution type as mapreduce and cluster name as local (joa23 via
gates).
PIG-106: Change StringBuffer and String '+' to StringBuilder (francisoud
via gates).
PIG-111: Reworked configuration to be setable via properties.
(Contributions from joa23, pi_song, and oae via gates).
BUG FIXES
PIG-24 Files that were incorrectly placed under test/reports have been
removed. ant clean now cleans test/reports. (milindb via gates)
PIG-25 com.yahoo.pig dir left under pig/test by mistake. removed it (olgan@)
PIG-23 Made pig work with java 1.5. (milindb via gates)
PIG-17 integrated with Hadoop 0.15 (olgan@)
PIG-33 Help was commented out - uncommented (olgan)
PIG-31: second half of concurrent mode problem addressed (olgan)
PIG-14: added heartbeat functionality (olgan)
PIG-17: updated hadoop15.jar to match hadoop 0.15.1 release
PIG-29: fixed bag factory to be properly initialized (utkarsh)
PIG-43: fixed problem where using the combiner prevented a pig alias
from being evaluated more than once. (gates)
PIG-45: Fixed pig.pl to not assume hodrc file is named the same as
cluster name (gates).
PIG-7 (more): Fixed bug in PigCombiner where it was writing IndexedTuples
instead of Tuples, causing Reducer to crash in some cases.
PIG-41: Added patterns to svn:ignore
PIG-51: Fixed combiner in the presence of flattening
PIG-61: Fixed MapreducePlanCompiler to use PigContext to load up the
comparator function instead of Class.forName. (gates)
PIG-63: Fix for non-ascii UTF-8 data (breed@ and olgan@)
PIG-77: Added eclipse specific files to svn:ignore
PIG-57: Fixed NPE in PigContext.fixUpDomain (francisoud via gates)
PIG-69: NPE in PigContext.setJobtrackerLocation (francisoud via gates)
PIG-78: src/org/apache/pig/builtin/PigStorage.java doesn't compile (arun
via olgan)
PIG-87: Fix pig.pl to find java via JAVA_HOME instead of hardcoded default
path. Also fix it to not die if pigclient.conf is missing. (craigm via
gates).
PIG-89: Fix DefaultDataBag, DistinctDataBag, SortedDataBag to close spill
files when they are done spilling (contributions by craigm, breed, and
gates, committed by gates).
PIG-95: Remove System.exit() statements from inside pig (joa23 via gates).
PIG-65: convert tabs to spaces (groves via olgan)
PIG-97: Turn off combiner in the case of Cogroup, as it doesn't work when
more than one bag is involved (gates).
PIG-92: Fix NullPointerException in PIgContext due to uninitialized conf
reference. (francisoud via gates)
PIG-80: In a number of places stack trace information was being lost by an
exception being caught, and a different exception then thrown. All those
locations have been changed so that the new exception now wraps the old.
(francisoud via gates).
PIG-84: Converted printStackTrace calls to calls to the logger.
(francisoud via gates).
PIG-88: Remove unused HadoopExe import from Main. (pi_song via gates).
PIG-99: Fix to make unit tests not run out of memory. (francisoud via
gates).
PIG-107: enabled several tests. (francisoud via olgan)
PIG-46: abort processing on error for non-interactive mode (olston via
olgan)
PIG-109: improved exception handling (oae via olgan)
PIG-72: Move unit tests to use MiniDFS and MiniMR so that unit tests can
be run w/o access to a hadoop cluster. (xuzh via gates)
PIG-68: improvements to build.xml (joa23 via olgan)
PIG-110: Replaced code accidently merged out in PIG-32 fix that handled
flattening the combiner case. (gates and oae)
PIG-68 broke the build process by hardwiring hadoop15 jar for the purpose
of compile. Fixed that (olgan)
PIG-124: only run one test (ant test -Dtestcase=TestMapReduce) not the
complete test suite (xuzh vi olgan)
PIG-127: changes to build.xml to have description for each target
(francisoud via olgan)
PIG-101: changes in tests to use enum type (francisoud via olgan)
PIG-125: Improve exception handling in cases when an attempt is made to
access a field as a tuple, and it turns out not to be a tuple (oae via
gates).
PIG-13: make the code use svn only if available (joa23 via olgan)
PIG-118: make sure union/join/cross takes 2 params (pi_song vi olgan)
PIG-94: M1 for streaming: maps and reduce side support with default
(de)serializer (acmurthy via olgan)
PIG-129: making sure that temp files are stored in task's home dir and
cleaned up
PIG-115: Removed Yahoo specific scripts/pig.pl, replaced with generic
bash script bin/pig. Moved startHOD.expect to bin (joa23 via gates).
PIG-18: changes to make pig work with Hadoop 0.16 and HOD 0.4 (olgan)
PIG-164: Fix memory issue in SpillableMemoryManager to partially clean the list of
bags each time a new bag is added rather than waiting until the garbage
collector tells us we are out of memory (gates).
PIG-154: moving parsing for DEFINE and STORE into QueryParser
PIG-100: improved error handling
PIG-94: changes for M2 of streaming: input/ouptut/ ship/cache error
handling
PIG-108: Fixed PigCombine to not do initialization on every call to
reduce, but instead only do it once in the call to configure. (joa23 via
gates).
PIG-172: dealing with NULL error messages in exceptions (olgan)
PIG-170: sort bags so that largest ones are released first
PIG-122: Added build and src-gen to the list of ignore files in
the top level directory (joa23 via gates).
PIG-94: M3 code update for streaming (arunc via olgan)
PIG-179: Changed PigRecordReader to be a static singleton rather than
thread local. (gates).
PIG-174,180: bug fixes in streaming (arunc via olgan)
PIG-181: streaming bug fixing (arunc via olgan)
PIG-182: streaming bug fix (arunc via olgan)
PIG-184: streaming bug fixes
PIG-153: Incorrect result caused by dump in between statements (pi_song
via gates).
PIG-178: Use of schema on secondary output of SPLIT throws
IndexOutOfBoundsException (kali via gates).
PIG-203: Fix bug in parameter substitution code where any pig script over
1k caused pig to freeze. (kali via gates)
PIG-204: Repair broken input splits (acmurthy via gates).
PIG-188: Fix mismatches between pig slicer changes and new streaming
feature (acmurthy via gates).
PIG-149, PIG-150: Fix doc target so that ant can generate docs (xuzh via
gates).
PIG-183: Catch when a UDF has been compiled with the wrong version of
java and give a RuntimeException (pi_song via gates).
PIG-114: store one alias/logicalPlan twice leads to instantiation of
StoreFunc as LoadFunc (pi_song via gates).
PIG-213: Remove non-static references to logger from data bags and tuples,
as it causes significant overhead (vgeschel via gates).
PIG-216: Fix streaming to work with commands that use unix pipes (acmurthy
via gates).
PIG-207: Fix illustrate command to work in mapreduce mode (shubhamc via
gates).
PIG-218: Fixed param generation to work with arbitrary commands
PIG-220: Fixed definition of parameter name for param substitution
PIG-151: fixes to code that handles bzip files
PIG-222: fix build break
PIG-226: fix for streaming optimization bug (acmurthy via olgan)
PIG-228: make multiple streaming outputs adhere to spec (acmurthy via olgan)
PIG-224: fix to error handling code to produce correct error code
PIG-176: Change bag spilling so that bags below a certain threshold are
not spilled, thus avoiding proliferation of small files (pi_song via
gates).
PIG-227: making load/store function optional in stream input/output spec
(acmurthy via olgan)
PIG-215: Cleanup a few dangling ends left by PIG-111 (pi_song via gates).
PIG-229: Proper error handling in case of deserializer failure
PIG-230: Handling shipment for multiple ship/cache commands (acmurthy via
olgan)
PIG-219: Change unit tests to run both local and map reduce modes
(kali via gates).
PIG-202: Fix Order by so that user provided comparator func is used for
quantile determination (kali via gates).
PIG-231: validation for ship, cache, and skippath (acmurthy via olgan)
PIG-232: fix for number of output records when BinaryStirage is used
(acmurthy via olgan)
PIG-232: fix for number of input records when BinaryStirage is used
(acmurthy via olgan)
PIG-232: let valid cache specifications through (acmurthy via olgan)
PIG-237: validation of the output directory (pi_song via olgan)
PIG-236: Fix properties so that values specified via the
command line (-D) are not ignored (pkamath via gates).
PIG-198: integration with hadoop 17 (acmurthy via olgan)
PIG-85: allowing control characters as delimiters for PigStorage (pi_song
via olgan)
PIG-250: disabling speculative execution (olgan)
PIG-250: re-enabling speculative execution and fixing the failure
(acmurthy via olgan)
PIG-85: memory optimization (pi_song via olgan)
PIG-243: Fixing unit tests on windows (daijy via olgan)
PIG-198: Fixed pig script to pick up hadoop 17 instead of 15 (pi_song via gates).
PIG-266: fix warnings caused by HOD (olgan)
PIG-245: added math functions to the piggybank (ajaygarg via olgan)
PIG-255: make non-default constructors work for algebraic functions
(ajaygarg via olgan)
PIG-272: problem with streaming and intermediate store (acmurthy via olgan)
PIG-243: make unit tests work on windows (daijy via olgan)
PIG-271: added tutorial to SVN (olgan)
PIG-235: memory management improvements (pkamath via olgan)
PIG-284: target for building source jar (oae via olgan)
PIG-34: added missing licenses
PIG-34: added LICENSE, NOTICE and README file
PIG-291: hod.param parameters not passed properly (thatha via olgan)
PIG-34: changes to build process to create distribution tar file
PIG-34: updated CHANGES.txt
PIG-472: Added RegExLoader to piggybank, an abstract loader class to parse
text files via regular espressions (spackest via gates)
PIG-473: Added CommonLogLoader, a subclass of RegExLoader to piggybank (spackest via gates)
PIG-474: Added MyRegexLoader, a subclass of RegExLoader, to piggybank (spackest via gates)