m202 week2 updated & 3 finished

m202/week2/hw2.4/1.log

@@ -0,0 +1,76 @@
+2014-05-11T08:55:48.507-0700 [initandlisten] MongoDB starting : pid=18765 port=27017 dbpath=/home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1 64-bit host=ubuntu
+2014-05-11T08:55:48.507-0700 [initandlisten] db version v2.6.1
+2014-05-11T08:55:48.507-0700 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
+2014-05-11T08:55:48.507-0700 [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
+2014-05-11T08:55:48.507-0700 [initandlisten] allocator: tcmalloc
+2014-05-11T08:55:48.507-0700 [initandlisten] options: { net: { port: 27017 }, processManagement: { fork: true }, replication: { oplogSizeMB: 64 }, storage: { dbPath: "1", smallFiles: true }, systemLog: { destination: "file", path: "1.log" } }
+2014-05-11T08:55:48.515-0700 [initandlisten] journal dir=/home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/journal
+2014-05-11T08:55:48.516-0700 [initandlisten] recover : no journal files present, no recovery needed
+2014-05-11T08:55:48.565-0700 [initandlisten] couldn't unlink socket file /tmp/mongodb-27017.sockerrno:1 Operation not permitted skipping
+2014-05-11T08:55:48.565-0700 [FileAllocator] allocating new datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/local.ns, filling with zeroes...
+2014-05-11T08:55:48.565-0700 [FileAllocator] creating directory /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp
+2014-05-11T08:55:48.567-0700 [FileAllocator] done allocating datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/local.ns, size: 16MB,  took 0 secs
+2014-05-11T08:55:48.568-0700 [FileAllocator] allocating new datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/local.0, filling with zeroes...
+2014-05-11T08:55:48.569-0700 [FileAllocator] done allocating datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/local.0, size: 16MB,  took 0.001 secs
+2014-05-11T08:55:48.570-0700 [initandlisten] build index on: local.startup_log properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.startup_log" }
+2014-05-11T08:55:48.570-0700 [initandlisten] 	 added index to empty collection
+2014-05-11T08:55:48.573-0700 [initandlisten] waiting for connections on port 27017
+2014-05-11T08:55:56.593-0700 [initandlisten] connection accepted from 127.0.0.1:44078 #1 (1 connection now open)
+2014-05-11T08:56:48.571-0700 [clientcursormon] mem (MB) res:45 virt:1189
+2014-05-11T08:56:48.571-0700 [clientcursormon]  mapped (incl journal view):992
+2014-05-11T08:56:48.571-0700 [clientcursormon]  connections:1
+2014-05-11T09:00:19.158-0700 [conn1] end connection 127.0.0.1:44078 (0 connections now open)
+2014-05-11T09:00:22.386-0700 [initandlisten] connection accepted from 127.0.0.1:44079 #2 (1 connection now open)
+2014-05-11T09:01:48.598-0700 [clientcursormon] mem (MB) res:46 virt:1190
+2014-05-11T09:01:48.598-0700 [clientcursormon]  mapped (incl journal view):992
+2014-05-11T09:01:48.598-0700 [clientcursormon]  connections:1
+2014-05-11T09:05:52.773-0700 [conn2] end connection 127.0.0.1:44079 (0 connections now open)
+2014-05-11T09:05:54.550-0700 [initandlisten] connection accepted from 127.0.0.1:44109 #3 (1 connection now open)
+2014-05-11T09:06:48.620-0700 [clientcursormon] mem (MB) res:47 virt:1190
+2014-05-11T09:06:48.620-0700 [clientcursormon]  mapped (incl journal view):992
+2014-05-11T09:06:48.620-0700 [clientcursormon]  connections:1
+2014-05-11T09:07:52.528-0700 [conn3] repairDatabase fragDB
+2014-05-11T09:07:52.529-0700 [conn3] repairDatabase fragDB
+2014-05-11T09:07:52.530-0700 [conn3] removeJournalFiles
+2014-05-11T09:07:52.538-0700 [FileAllocator] allocating new datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.ns, filling with zeroes...
+2014-05-11T09:07:52.538-0700 [FileAllocator] creating directory /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/_tmp
+2014-05-11T09:07:52.540-0700 [FileAllocator] done allocating datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.ns, size: 16MB,  took 0 secs
+2014-05-11T09:07:52.542-0700 [FileAllocator] allocating new datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.0, filling with zeroes...
+2014-05-11T09:07:52.545-0700 [FileAllocator] done allocating datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.0, size: 16MB,  took 0 secs
+2014-05-11T09:07:52.849-0700 [FileAllocator] allocating new datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.1, filling with zeroes...
+2014-05-11T09:07:52.854-0700 [FileAllocator] done allocating datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.1, size: 32MB,  took 0.004 secs
+2014-05-11T09:07:54.344-0700 [FileAllocator] allocating new datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.2, filling with zeroes...
+2014-05-11T09:07:54.374-0700 [FileAllocator] done allocating datafile /home/jrodriguez/Projects/mongocourses/m202/week2/homework/hw2.4/1/_tmp_repairDatabase_0/fragDB.2, size: 64MB,  took 0.028 secs
+2014-05-11T09:07:55.076-0700 [conn3] removeJournalFiles
+2014-05-11T09:07:55.091-0700 [conn3] restarting 0 index build(s)
+2014-05-11T09:07:55.092-0700 [conn3] command fragDB.$cmd command: repairDatabase { repairDatabase: 1.0 } keyUpdates:0 numYields:0 locks(micros) W:2562449 reslen:37 2562ms
+2014-05-11T09:08:51.096-0700 [clientcursormon] mem (MB) res:90 virt:590
+2014-05-11T09:08:51.096-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:08:51.096-0700 [clientcursormon]  connections:1
+2014-05-11T09:09:37.587-0700 [conn3] end connection 127.0.0.1:44109 (0 connections now open)
+2014-05-11T09:09:51.102-0700 [clientcursormon] mem (MB) res:105 virt:590
+2014-05-11T09:09:51.102-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:09:51.102-0700 [clientcursormon]  connections:0
+2014-05-11T09:10:18.711-0700 [initandlisten] connection accepted from 127.0.0.1:44112 #4 (1 connection now open)
+2014-05-11T09:10:18.947-0700 [conn4] command fragDB.$cmd command: count { count: "fragColl", fields: null, query: { string: "testStringForPadding0000000000000000000000000000000000000000" } } keyUpdates:0 numYields:0 locks(micros) r:173849 reslen:48 173ms
+2014-05-11T09:10:19.005-0700 [conn4] end connection 127.0.0.1:44112 (0 connections now open)
+2014-05-11T09:10:25.824-0700 [initandlisten] connection accepted from 127.0.0.1:44113 #5 (1 connection now open)
+2014-05-11T09:10:25.911-0700 [conn5] end connection 127.0.0.1:44113 (0 connections now open)
+2014-05-11T09:10:51.114-0700 [clientcursormon] mem (MB) res:154 virt:590
+2014-05-11T09:10:51.114-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:10:51.114-0700 [clientcursormon]  connections:0
+2014-05-11T09:15:51.144-0700 [clientcursormon] mem (MB) res:154 virt:590
+2014-05-11T09:15:51.144-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:15:51.144-0700 [clientcursormon]  connections:0
+2014-05-11T09:20:51.168-0700 [clientcursormon] mem (MB) res:151 virt:590
+2014-05-11T09:20:51.168-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:20:51.168-0700 [clientcursormon]  connections:0
+2014-05-11T09:25:51.192-0700 [clientcursormon] mem (MB) res:151 virt:590
+2014-05-11T09:25:51.192-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:25:51.192-0700 [clientcursormon]  connections:0
+2014-05-11T09:30:51.217-0700 [clientcursormon] mem (MB) res:151 virt:590
+2014-05-11T09:30:51.217-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:30:51.217-0700 [clientcursormon]  connections:0
+2014-05-11T09:35:51.242-0700 [clientcursormon] mem (MB) res:151 virt:590
+2014-05-11T09:35:51.242-0700 [clientcursormon]  mapped (incl journal view):320
+2014-05-11T09:35:51.242-0700 [clientcursormon]  connections:0

m202/week2/hw2.4/1/mongod.lock

@@ -0,0 +1 @@
+18765

m202/week2/hw2.4/2/mongod.lock

@@ -0,0 +1 @@
+17726

m202/week2/hw2.4/3/mongod.lock

@@ -0,0 +1 @@
+17773

m202/week2/hw2.4/README.md

@@ -0,0 +1,56 @@
+# Homework 2.4 Reclaiming disk space
+
+Note: MongoProc validation for this exercise requires that you are running MongoDB 2.6
+
+Over time a MongoDB database can become fragmented and cause reduced storage efficiency.
+
+In this problem we have provided you with an already fragmented database. You will want to extract the given handout and set mongod's dbpath to the appropriate folder.
+
+Your goal in this homework is to increase storage efficiency and reclaim disk space. You may use any method learned thus far to accomplish this goal. However, please do not tamper (remove/update/insert) with the database given apart from querying it to find out more information about the fragmented components.
+
+You are encouraged to use MongoProc to continuously test your progress until you are ready to grade the homework.
+
+Useful Formulas
+
+Collection Storage Efficiency:
+db.collection.stats().size/db.collection.stats().storageSize
+Database Storage Efficiency:
+(db.stats().dataSize + db.stats().indexSize) / db.stats().fileSize
+
+# Solution
+Database: fragDB
+Command: db.repairDatabase()
+- Before:
+db.fragColl.stats().size/db.fragColl.stats().storageSize
+0.14289938663772606
+(db.stats().dataSize + db.stats().indexSize) / db.stats().fileSize
+0.12821715218680246
+- After:
+0.8213309503784693
+0.45167418888636995
+
+# Example javascript to do the task and get info about fragmentation
+$ mongo fragDB task.js
+
+// Get a the current collection size.
+var storage = db.fragColl.storageSize();
+var total = db.fragColl.totalSize();
+
+print('Storage Size: ' + tojson(storage));
+
+print('TotalSize: ' + tojson(total));
+
+print('-----------------------');
+print('Running db.repairDatabase()');
+print('-----------------------');
+
+// Run repair
+db.repairDatabase()
+
+// Get new collection sizes.
+var storage_a = db.fragColl.storageSize();
+var total_a = db.fragColl.totalSize();
+
+print('Storage Size: ' + tojson(storage_a));
+print('TotalSize: ' + tojson(total_a));
+

m202/week2/hw2.4/init_config.js

@@ -0,0 +1,7 @@
+config = { _id: "m202", members:[
+          { _id : 0, host : "localhost:27017"},
+          { _id : 1, host : "localhost:27018"},
+          { _id : 2, host : "localhost:27019"} ]
+	  };
+rs.initiate(config);
+rs.status()

m202/week2/hw2.4/init_instance.sh

@@ -0,0 +1,2 @@
+
+mongod --logpath "1.log" --dbpath 1 --port 27017 --smallfiles --oplogSize 64 --fork

m202/week2/hw2.4/init_instances.sh

@@ -0,0 +1,3 @@
+mongod --replSet m202 --logpath "1.log" --dbpath 1 --port 27017 --smallfiles --oplogSize 64 --fork 
+mongod --replSet m202 --logpath "2.log" --dbpath 2 --port 27018 --smallfiles --oplogSize 64 --fork
+mongod --replSet m202 --logpath "3.log" --dbpath 3 --port 27019 --smallfiles --oplogSize 64 --fork

m202/homework/hw3.4/README.md → m202/week3/homework/hw3.4/README.md

@@ -8,5 +8,5 @@ Which of the following are steps you should take in order for this to be success
 The writes that occurred before the backup will cause problems if replayed, because they will cause the database to deviate from the state it was in before the mongodump, so they need to be excised from the replay.
 The mongodump will be repeated when the oplog is replayed, potentially overwriting the original mongodump, so that operation needs to be removed from the oplog.
 The writes after the mongodump will need to be sorted, as they may not be in order.
-The db.collection.drop() command will still be in the oplog, and will need to be avoided.
-Ensure that any writes that occurred after the drop() command are not replayed (you might need to deal with these later), because these might lead to unexpected or inconsistent results.
+(OK) The db.collection.drop() command will still be in the oplog, and will need to be avoided.
+(OK) Ensure that any writes that occurred after the drop() command are not replayed (you might need to deal with these later), because these might lead to unexpected or inconsistent results.