-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path2020-04-08-000-xray-and-wai.html
170 lines (152 loc) · 16.4 KB
/
2020-04-08-000-xray-and-wai.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="alternate"
type="application/rss+xml"
href="https://magnus.therning.org/feed.xml"
title="RSS feed for https://magnus.therning.org/">
<title>X-Ray and WAI</title>
<meta name="author" content="Magnus Therning"><meta name="referrer" content="no-referrer"><link href= "static/style.css" rel="stylesheet" type="text/css" /><link href= "static/htmlize.css" rel="stylesheet" type="text/css" /><link href= "static/extra_style.css" rel="stylesheet" type="text/css" /></head>
<body>
<div id="preamble" class="status"><div class="nav-bar"><a class="nav-link" href="./index.html">Top</a><a class="nav-link" href="./archive.html">Archive</a><a class="nav-link align-right" href="./feed.xml"><img src="static/rss-feed-icon.png" style="height: 24px;" /></a></div></div>
<div id="content">
<div class="post-date">08 Apr 2020</div><h1 class="post-title"><a href="https://magnus.therning.org/2020-04-08-000-xray-and-wai.html">X-Ray and WAI</a></h1>
<p>
For a while we've been planning on introducing <a href="https://aws.amazon.com/xray/">AWS X-Ray</a> into our system at
work. There's official support for a few languages, but not too surprisingly
Haskell isn't on that list. I found <a href="https://github.com/freckle/aws-xray-client">freckle/aws-xray-client</a> on GitHub, which is
so unofficial that it isn't even published on Hackage. While it looks very good,
I suspect it does more than I need and since it lacks licensing information I
decided to instead implement a version tailored to our needs.
</p>
<p>
As a first step I implemented a WAI <i>middleware</i> that wraps an HTTP request and
reports the time it took to produce a response. Between the <a href="https://docs.aws.amazon.com/xray/latest/devguide/aws-xray.html">X-Ray Developer
Guide</a> and the code in Freckle's git repo it turned out to be fairly simple.
</p>
<p>
First off, this is the first step towards X-Ray nirvana, so all I'm aiming for
is minimal support. That means all I want is to send <a href="https://docs.aws.amazon.com/xray/latest/devguide/xray-api-segmentdocuments.html#api-segmentdocuments-fields">minimal X-Ray segment</a>s,
with the small addition that I want to support <code>parent_id</code> from the start.
</p>
<p>
The first step then is to parse the HTTP header containing the X-Ray information
– <a href="https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-tracingheader"><code>X-Amzn-Trace-Id</code></a>. For now I'm only interested in two parts, <code>Root</code> and
<code>Parent</code>, so for simplicity's sake I use a tuple to keep them in. The idea is to
take the header's value, split on <code>;</code> to get the parts, then split each part in
two, a key and a value, and put them into an association list (<code>[(Text, Text)]</code>)
for easy lookup using, well <code>lookup</code>.
</p>
<div class="org-src-container">
<pre class="src src-haskell"><span class="org-haskell-definition">parseXRayTraceIdHdr</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">Text</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Maybe</span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-type">Text</span>, <span class="org-haskell-type">Maybe</span> <span class="org-haskell-type">Text</span><span class="org-rainbow-delimiters-depth-1">)</span>
<span class="org-haskell-definition">parseXRayTraceIdHdr</span> hdr <span class="org-haskell-operator">=</span> <span class="org-haskell-keyword">do</span>
bits <span class="org-haskell-operator"><-</span> traverse parseHeaderComponent <span class="org-haskell-operator">$</span> T.split <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-operator">==</span> <span class="org-string">';'</span><span class="org-rainbow-delimiters-depth-1">)</span> hdr
traceId <span class="org-haskell-operator"><-</span> lookup <span class="org-string">"Root"</span> bits
<span class="org-haskell-keyword">let</span> parent <span class="org-haskell-operator">=</span> lookup <span class="org-string">"Parent"</span> bits
pure <span class="org-rainbow-delimiters-depth-1">(</span>traceId, parent<span class="org-rainbow-delimiters-depth-1">)</span>
<span class="org-haskell-definition">parseHeaderComponent</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">Text</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Maybe</span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-type">Text</span>, <span class="org-haskell-type">Text</span><span class="org-rainbow-delimiters-depth-1">)</span>
<span class="org-haskell-definition">parseHeaderComponent</span> cmp <span class="org-haskell-operator">=</span> <span class="org-haskell-keyword">case</span> T.split <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-operator">==</span> <span class="org-string">'='</span><span class="org-rainbow-delimiters-depth-1">)</span> cmp <span class="org-haskell-keyword">of</span>
<span class="org-rainbow-delimiters-depth-1">[</span>name, value<span class="org-rainbow-delimiters-depth-1">]</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-constructor">Just</span> <span class="org-rainbow-delimiters-depth-1">(</span>name, value<span class="org-rainbow-delimiters-depth-1">)</span>
<span class="org-haskell-keyword">_</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-constructor">Nothing</span>
</pre>
</div>
<p>
The start and end times for processing a request are also required. The docs say
that using at least millisecond resolution is a good idea, so I decided to do
exactly that. <code>NominalDiffTime</code>, which is what <code>getPOSIXTime</code> produces, supports
a resolution of picoseconds (though I doubt my system's clock does) which
requires a bit of (type-based) converting.
</p>
<div class="org-src-container">
<pre class="src src-haskell"><span class="org-haskell-definition">mkTimeInMilli</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">IO</span> <span class="org-haskell-type">Milli</span>
<span class="org-haskell-definition">mkTimeInMilli</span> <span class="org-haskell-operator">=</span> ndfToMilli <span class="org-haskell-operator"><$></span> getPOSIXTime
<span class="org-haskell-keyword">where</span>
ndfToMilli <span class="org-haskell-operator">=</span> fromRational <span class="org-haskell-operator">.</span> toRational
</pre>
</div>
<p>
The last support function needed is one that creates the segment. Just
building the JSON object, using <i>aeson</i>'s <code>object</code>, is enough at this
point.
</p>
<div class="org-src-container">
<pre class="src src-haskell"><span class="org-haskell-definition">mkSegment</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">Text</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Text</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Milli</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Milli</span> <span class="org-haskell-operator">-></span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-type">Text</span>, <span class="org-haskell-type">Maybe</span> <span class="org-haskell-type">Text</span><span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Value</span>
<span class="org-haskell-definition">mkSegment</span> name id startTime endTime <span class="org-rainbow-delimiters-depth-1">(</span>root, parent<span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-haskell-operator">=</span>
object <span class="org-haskell-operator">$</span> <span class="org-rainbow-delimiters-depth-1">[</span> <span class="org-string">"name"</span> <span class="org-haskell-operator">.=</span> name
, <span class="org-string">"id"</span> <span class="org-haskell-operator">.=</span> id
, <span class="org-string">"trace_id"</span> <span class="org-haskell-operator">.=</span> root
, <span class="org-string">"start_time"</span> <span class="org-haskell-operator">.=</span> startTime
, <span class="org-string">"end_time"</span> <span class="org-haskell-operator">.=</span> endTime
<span class="org-rainbow-delimiters-depth-1">]</span> <span class="org-haskell-operator"><></span> p
<span class="org-haskell-keyword">where</span>
p <span class="org-haskell-operator">=</span> maybe <span class="org-haskell-constructor"><span class="org-rainbow-delimiters-depth-1">[]</span></span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-operator">\</span> v <span class="org-haskell-operator">-></span> <span class="org-rainbow-delimiters-depth-2">[</span><span class="org-string">"parent_id"</span> <span class="org-haskell-operator">.=</span> v<span class="org-rainbow-delimiters-depth-2">]</span><span class="org-rainbow-delimiters-depth-1">)</span> parent
</pre>
</div>
<p>
Armed with all this, I can now put together a WAI middleware that
</p>
<ol class="org-ol">
<li>records the start time of the call</li>
<li>processes the request</li>
<li>sends off the response and keeps the result of it</li>
<li>records the end time</li>
<li>parses the tracing header</li>
<li>builds the segment prepended with the <a href="https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html#xray-api-daemon">X-Ray daemon header</a></li>
<li>sends the segment to the X-Ray daemon</li>
</ol>
<div class="org-src-container">
<pre class="src src-haskell"><span class="org-haskell-definition">traceId</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">Text</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-type">Middleware</span>
<span class="org-haskell-definition">traceId</span> xrayName app req sendResponse <span class="org-haskell-operator">=</span> <span class="org-haskell-keyword">do</span>
startTime <span class="org-haskell-operator"><-</span> mkTimeInMilli
app req <span class="org-haskell-operator">$</span> <span class="org-haskell-operator">\</span> res <span class="org-haskell-operator">-></span> <span class="org-haskell-keyword">do</span>
rr <span class="org-haskell-operator"><-</span> sendResponse res
endTime <span class="org-haskell-operator"><-</span> mkTimeInMilli
theId <span class="org-haskell-operator"><-</span> T.pack <span class="org-haskell-operator">.</span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-operator">\</span> v <span class="org-haskell-operator">-></span> showHex v <span class="org-string">""</span><span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-haskell-operator"><$></span> randomIO <span class="org-haskell-operator">@</span><span class="org-haskell-constructor">Word64</span>
<span class="org-haskell-keyword">let</span> traceParts <span class="org-haskell-operator">=</span> <span class="org-rainbow-delimiters-depth-1">(</span>decodeUtf8 <span class="org-haskell-operator"><$></span> requestHeaderTraceId req<span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-haskell-operator">>>=</span> parseXRayTraceIdHdr
segment <span class="org-haskell-operator">=</span> mkSegment xrayName theId startTime endTime <span class="org-haskell-operator"><$></span> traceParts
<span class="org-haskell-keyword">case</span> segment <span class="org-haskell-keyword">of</span>
<span class="org-haskell-constructor">Nothing</span> <span class="org-haskell-operator">-></span> pure <span class="org-haskell-constructor"><span class="org-rainbow-delimiters-depth-1">()</span></span>
<span class="org-haskell-constructor">Just</span> segment' <span class="org-haskell-operator">-></span> sendXRayPayload <span class="org-haskell-operator">$</span> toStrict <span class="org-haskell-operator">$</span> prepareXRayPayload segment'
pure rr
<span class="org-haskell-keyword">where</span>
prepareXRayPayload segment <span class="org-haskell-operator">=</span>
<span class="org-haskell-keyword">let</span> header <span class="org-haskell-operator">=</span> object <span class="org-rainbow-delimiters-depth-1">[</span><span class="org-string">"format"</span> <span class="org-haskell-operator">.=</span> <span class="org-rainbow-delimiters-depth-2">(</span><span class="org-string">"json"</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">String</span><span class="org-rainbow-delimiters-depth-2">)</span>, <span class="org-string">"version"</span> <span class="org-haskell-operator">.=</span> <span class="org-rainbow-delimiters-depth-2">(</span><span class="org-highlight-numbers-number">1</span> <span class="org-haskell-operator">::</span> <span class="org-haskell-type">Int</span><span class="org-rainbow-delimiters-depth-2">)</span><span class="org-rainbow-delimiters-depth-1">]</span>
<span class="org-haskell-keyword">in</span> encode header <span class="org-haskell-operator"><></span> <span class="org-string">"\n"</span> <span class="org-haskell-operator"><></span> encode segment
sendXRayPayload payload <span class="org-haskell-operator">=</span> <span class="org-haskell-keyword">do</span>
addrInfos <span class="org-haskell-operator"><-</span> S.getAddrInfo <span class="org-haskell-constructor">Nothing</span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-constructor">Just</span> <span class="org-string">"127.0.0.1"</span><span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-rainbow-delimiters-depth-1">(</span><span class="org-haskell-constructor">Just</span> <span class="org-string">"2000"</span><span class="org-rainbow-delimiters-depth-1">)</span>
<span class="org-haskell-keyword">case</span> addrInfos <span class="org-haskell-keyword">of</span>
<span class="org-haskell-constructor"><span class="org-rainbow-delimiters-depth-1">[]</span></span> <span class="org-haskell-operator">-></span> pure <span class="org-haskell-constructor"><span class="org-rainbow-delimiters-depth-1">()</span></span> <span class="org-comment-delimiter">-- </span><span class="org-comment">silently skip</span>
<span class="org-rainbow-delimiters-depth-1">(</span>xrayAddr<span class="org-haskell-constructor">:</span><span class="org-haskell-keyword">_</span><span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-haskell-operator">-></span> <span class="org-haskell-keyword">do</span>
sock <span class="org-haskell-operator"><-</span> S.socket <span class="org-rainbow-delimiters-depth-1">(</span>S.addrFamily xrayAddr<span class="org-rainbow-delimiters-depth-1">)</span> <span class="org-haskell-constructor">S.Datagram</span> S.defaultProtocol
S.connect sock <span class="org-rainbow-delimiters-depth-1">(</span>S.addrAddress xrayAddr<span class="org-rainbow-delimiters-depth-1">)</span>
sendAll sock payload
S.close sock
</pre>
</div>
<p>
The next step will be to instrument the actual processing. The service I'm
instrumenting is asynchronous, so all the work happens <i>after</i> the response has
been sent. My plan for this is to use subsegments to record it. That means I'll
have to
</p>
<ul class="org-ul">
<li>keep the <code>Root</code> and ID (<code>theId</code> in <code>traceId</code> above) for use in subsegments</li>
<li>keep the original tracing header, for use in outgoing calls</li>
<li>make sure all outgoing HTTP calls include a tracing header with a proper
<code>Parent</code></li>
<li>wrap all outgoing HTTP calls with time keeping and sending a subsegment to the
X-Ray daemon</li>
</ul>
<p>
I'm saving that work for a rainy day though, or rather, for a day when I'm so
upset at Clojure that I don't want to see another parenthesis.
</p>
<p>
<i>Edit (2020-04-10):</i> Corrected the segment field name for the parent ID, it
should be <code>parent_id</code>.
</p>
<div class="taglist"><a href="https://magnus.therning.org/tags.html">Tags</a>: <a href="https://magnus.therning.org/tag-aws.html">AWS</a> <a href="https://magnus.therning.org/tag-haskell.html">haskell</a> <a href="https://magnus.therning.org/tag-xray.html">XRay</a> </div></div>
<div id="postamble" class="status"><!-- org-static-blog-page-postamble --></div>
</body>
</html>