1

extends layout

block content

	.container-fluid
		.row

			.col-md-8
				h3 Data Discovery  
				p Logscape makes it easy to work with unstructured data. Logscape learns the structure of your data as it is importing it using the Key-Value Discovery feature. This features detects the presence of common valuepairs that exist in widespread formats such as xml and json. A wide range of key value types are supported by Logscape. 

				.image
					img(src="/images/searching-kv-fields.png")


				p  It happens automatically without any intervention from the user. 

				h3 Configuration
				p  Data Discovery happens at index time and is configured on the Data Source for the data. See 
					a(href="/ds-intelligent_field_discovery.html") Data Source Config Page  
					| for more. 

				h3 Facets and Key  Values 

				p Any discovered data is listed under the discovered fields and can be searched using the UI or using Logscape Search Syntax directly. 

				h4 Example of XML Exception in a log4j 
	
				p Some applications will log errors wrapped in an XML snippet 

					pre Feb 23 18:39:01 occular-sentinel CRON[6249]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime)) < error id="0x00023" host="10.28.1.170" type="Exception">
						| java.io.FileNotFoundException: fred.txt
						| at java.io.FileInputStream.<init>(FileInputStream.java)
						| at java.io.FileInputStream.<init>(FileInputStream.java)
						| at ExTest.readMyFile(ExTest.java:19)
						| at ExTest.main(ExTest.java:7)
						| &lt;/error>
						| Feb 23 18:40:39 occular-sentinel whoopsie[1124]: online

				p The XML attributes 
					em id, 
					em host  
					| and 
					em type 
					| will be extracted as system fields and will be searchable. 



				h4 json embedded in a web resource

				p JSON is growing as an application data exchange format and it is not uncommon to see log files embedding json results along side log messages. 

					pre 2014-04-12 INFO [requestor-0] Result  {
						|  	"collection" : {
						|  	"title" : "Blog",
						| 	"description" : "This is a description of my blog.",
						| 	"categories" : [ "Category-1", "Category-2" ]
						|  }
						|  }

				p The attributes: 
					em title, 
					em description 
					| and 
					em categories 
					| will be extracted. Collection will not be extracted because its value is another json object.