-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.xml
3939 lines (3835 loc) · 787 KB
/
index.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>CMA</title>
<link>https://kuklis.github.io/cma/</link>
<description>Recent content on CMA</description>
<generator>Hugo -- gohugo.io</generator>
<copyright>Copyright © 2021, Krisztian Kuklis; all rights reserved.</copyright>
<lastBuildDate>Sun, 21 Jul 2024 00:00:00 +0000</lastBuildDate><atom:link href="https://kuklis.github.io/cma/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Processing PowerShell Script Result by Orchestrator</title>
<link>https://kuklis.github.io/cma/post/vro8-powershell-result/</link>
<pubDate>Sun, 21 Jul 2024 00:00:00 +0000</pubDate>
<guid>https://kuklis.github.io/cma/post/vro8-powershell-result/</guid>
<description>
<p>Collecting and parsing PowerShell script result by Aria Automation Orchestrator.</p>
<div class="toc">
<H3>Table of Contents</H3>
<nav id="TableOfContents">
<ul>
<li><a href="#subject">Subject</a></li>
<li><a href="#open-and-reuse-a-session">Open (and reuse) a session</a></li>
<li><a href="#getting-the-output">Getting the output</a></li>
<li><a href="#get-results-as-json">Get Results as JSON</a></li>
<li><a href="#get-results-as-xml">Get Results as XML</a></li>
<li><a href="#hair-of-the-dog-xml-to-json">Hair of the dog: XML to JSON</a></li>
<li><a href="#wrap-up">Wrap up</a></li>
</ul>
</nav>
</div>
<h2 id="subject">Subject</h2>
<p>Recently I've been tasked to automate existing PowerShell script running. The scripts manage email messages on an Exchange Online server and I needed to collect and send the script output. I started to experiment with Orchestartor PowerShell plugin and came up with a solution presented below.</p>
<h2 id="open-and-reuse-a-session">Open (and reuse) a session</h2>
<p>In case you run multiple scripts on a single PowerShell host, it is worth to open a session and run the commands via this single session. This will save a lot of time, as creating a session can be time-comsuming (compared to the commands/scripts run on the host). The first sample workflow <tt>Open Session</tt> demonstrates the point. It runs two simple scripts, the first is the <tt>whoami</tt> command, and second is listing the local groups defined: <tt>Get-LocalGroup | Select-Object Name,SID</tt>. The test run is below: it took 5 seconds to build the connection, and less than a second to run each command:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vro8-powershell-result/open_session.png" alt=""></p>
<h2 id="getting-the-output">Getting the output</h2>
<p>The easiest way of getting the result is getting the host output (used in the <tt>Open Session</tt> workflow):</p>
<div class="highlight"><pre class="chroma"><code class="language-powershell" data-lang="powershell"><span class="ln">1</span><span class="n">var</span> <span class="n">session</span> <span class="p">=</span> <span class="n">psHost</span><span class="p">.</span><span class="n">getSession</span><span class="p">(</span><span class="n">sessionId</span><span class="p">);</span>
<span class="ln">2</span>
<span class="ln">3</span><span class="n">var</span> <span class="n">result</span> <span class="p">=</span> <span class="n">session</span><span class="p">.</span><span class="n">invokeScript</span><span class="p">(</span><span class="n">script1</span><span class="p">);</span>
<span class="ln">4</span>
<span class="ln">5</span><span class="n">System</span><span class="p">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&#34;Script run: &#34;</span> <span class="p">+</span> <span class="n">result</span><span class="p">.</span><span class="n">getInvocationState</span><span class="p">());</span>
<span class="hl"><span class="ln">6</span><span class="n">System</span><span class="p">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&#34;Host output:\n&#34;</span> <span class="p">+</span> <span class="n">result</span><span class="p">.</span><span class="n">getHostOutput</span><span class="p">());</span>
</span><span class="ln">7</span><span class="n">var</span> <span class="n">errors</span> <span class="p">=</span> <span class="n">result</span><span class="p">.</span><span class="n">getErrors</span><span class="p">();</span>
<span class="ln">8</span><span class="k">if</span> <span class="p">(</span><span class="n">errors</span><span class="p">.</span><span class="n">length</span><span class="p">)</span> <span class="n">System</span><span class="p">.</span><span class="n">error</span><span class="p">(</span><span class="s2">&#34;Script errors:\n&#34;</span> <span class="p">+</span> <span class="n">errors</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="s1">&#39;\n&#39;</span><span class="p">));</span></code></pre></div>
<p>There is a catch with this approach: each command can have a different output format, and we rely on string processing to gather the results. Here are the output of running the two scripts above:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vro8-powershell-result/host_output.png" alt=""></p>
<h2 id="get-results-as-json">Get Results as JSON</h2>
<p>This workflow use the <tt>getResults()</tt> function of the <tt>PowerShellInvocationResult</tt> object. This function returns a <tt>PowerShellRemotePSObject</tt>. This object has a getAsJson() function.</p>
<div class="highlight"><pre class="chroma"><code class="language-powershell" data-lang="powershell"><span class="ln">1</span><span class="k">if</span> <span class="p">(</span><span class="s2">&#34;Completed&#34;</span> <span class="p">==</span> <span class="n">result</span><span class="p">.</span><span class="n">getInvocationState</span><span class="p">())</span> <span class="p">{</span>
<span class="ln">2</span> <span class="n">var</span> <span class="n">json</span> <span class="p">=</span> <span class="n">result</span><span class="p">.</span><span class="n">getResults</span><span class="p">().</span><span class="n">getAsJson</span><span class="p">();</span>
<span class="ln">3</span> <span class="n">var</span> <span class="n">resultObj</span> <span class="p">=</span> <span class="n">JSON</span><span class="p">.</span><span class="n">parse</span><span class="p">(</span><span class="n">json</span><span class="p">);</span>
<span class="ln">4</span> <span class="n">System</span><span class="p">.</span><span class="n">log</span><span class="p">(</span><span class="n">JSON</span><span class="p">.</span><span class="n">stringify</span><span class="p">(</span><span class="n">resultObj</span><span class="p">,</span> <span class="n">null</span><span class="p">,</span> <span class="n">2</span><span class="p">));</span>
<span class="ln">5</span><span class="p">}</span>
</code></pre></div><p>Let's see what is the return value of this function for the following script: <tt>Get-LocalGroup -Name Guests; Get-LocalUser Guest</tt>:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vro8-powershell-result/result_json.png" alt=""></p>
<p>Does not seem to be easily parsed, unfortunately.</p>
<h2 id="get-results-as-xml">Get Results as XML</h2>
<p>Let's try the XML output: what could possibly go wrong?</p>
<div class="highlight"><pre class="chroma"><code class="language-powershell" data-lang="powershell"><span class="ln">1</span><span class="k">if</span> <span class="p">(</span><span class="s2">&#34;Completed&#34;</span> <span class="p">==</span> <span class="n">result</span><span class="p">.</span><span class="n">getInvocationState</span><span class="p">())</span> <span class="p">{</span>
<span class="ln">2</span> <span class="n">var</span> <span class="n">xml</span> <span class="p">=</span> <span class="n">result</span><span class="p">.</span><span class="n">getResults</span><span class="p">().</span><span class="n">getXml</span><span class="p">();</span>
<span class="ln">3</span> <span class="n">var</span> <span class="n">xmlObject</span> <span class="p">=</span> <span class="n">new</span> <span class="n">XML</span><span class="p">(</span><span class="n">xml</span><span class="p">);</span>
<span class="ln">4</span> <span class="n">System</span><span class="p">.</span><span class="n">log</span><span class="p">(</span><span class="n">xmlObject</span><span class="p">);</span>
<span class="ln">5</span><span class="p">}</span>
</code></pre></div><p>This is the same complex structure in XML. Hard to find the exact information we need within:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vro8-powershell-result/result_xml.png" alt=""></p>
<h2 id="hair-of-the-dog-xml-to-json">Hair of the dog: XML to JSON</h2>
<p>Let's process the the PowerShell script result by PowerShell!<br>Orchestrator has built-in PowerShell engine, and luckily we have the <tt>DeSerialize</tt> method in <a href="https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.psserializer">PSSerializer Class</a> to deserialize a PowerShell CliXml into an object. Then we convert the object to JSON and collect the result:</p>
<div class="highlight"><pre class="chroma"><code class="language-powershell" data-lang="powershell"><span class="ln"> 1</span><span class="k">function</span> <span class="n">Handler</span><span class="p">(</span><span class="nv">$context</span><span class="p">,</span> <span class="nv">$inputs</span><span class="p">)</span> <span class="p">{</span>
<span class="ln"> 2</span> <span class="nv">$xml</span> <span class="p">=</span> <span class="nv">$inputs</span><span class="p">.</span><span class="n">xml</span>
<span class="ln"> 3</span> <span class="nb">Write-Host</span> <span class="s2">&#34;XML input length:&#34;</span> <span class="nv">$xml</span><span class="p">.</span><span class="n">Length</span>
<span class="ln"> 4</span>
<span class="ln"> 5</span> <span class="k">try</span> <span class="p">{</span> <span class="nv">$json</span> <span class="p">=</span> <span class="no">[System.Management.Automation.PSSerializer]</span><span class="p">::</span><span class="n">DeSerialize</span><span class="p">(</span><span class="nv">$xml</span><span class="p">)</span> <span class="p">|</span> <span class="nb">ConvertTo-Json</span> <span class="n">-Compress</span> <span class="p">}</span>
<span class="ln"> 6</span> <span class="k">catch</span> <span class="p">{</span><span class="nb">Write-Error</span> <span class="nv">$_</span><span class="p">;</span> <span class="nv">$json</span> <span class="p">=</span> <span class="s1">&#39;{}&#39;</span><span class="p">}</span>
<span class="ln"> 7</span>
<span class="ln"> 8</span> <span class="nv">$output</span> <span class="p">=</span> <span class="p">@{</span><span class="n">json</span> <span class="p">=</span> <span class="nv">$json</span><span class="p">}</span>
<span class="ln"> 9</span> <span class="k">return</span> <span class="nv">$output</span>
<span class="ln">10</span><span class="p">}</span>
</code></pre></div><p>The collected result can be easily used:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vro8-powershell-result/result_xml_json.png" alt=""></p>
<h2 id="wrap-up">Wrap up</h2>
<p>PowerShell script result can be a complex object, and parsing the JSON/XML representations of it is complex. The method above provides a simple way to do that.</p>
<p>Bear in mind that the polyglot PowerShell engine is available only within Orchestrator intances licenced with Aria Automation.</p>
<p>You can find the sample workflows at GitHub: <a href="https://github.com/kuklis/vro8-packages">https://github.com/kuklis/vro8-packages</a><br>
Package com.test.powershell</p>
</description>
</item>
<item>
<title>Aria Automation Custom Resource Types - Update Custom Properties</title>
<link>https://kuklis.github.io/cma/post/vra8-custom-resources-4/</link>
<pubDate>Wed, 03 Jul 2024 00:00:00 +0000</pubDate>
<guid>https://kuklis.github.io/cma/post/vra8-custom-resources-4/</guid>
<description>
<p>How to update custom properties in deployments of a custom resource type</p>
<div class="toc">
<H3>Table of Contents</H3>
<nav id="TableOfContents">
<ul>
<li><a href="#deployment-resources">Deployment resources</a></li>
<li><a href="#recap-of-the-previous-posts">Recap of the previous posts</a></li>
<li><a href="#virtual-machine-folder-as-a-custom-resource-type">Virtual Machine folder as a Custom Resource Type</a>
<ul>
<li><a href="#create-vm-folder-workflow">Create VM folder workflow</a></li>
<li><a href="#update-vm-folder-delete-vm-folder-workflows">Update VM folder, Delete VM folder workflows</a></li>
</ul>
</li>
<li><a href="#define-vmfolder-custom-resource-type">Define VMfolder Custom Resource Type</a></li>
<li><a href="#cloud-template">Cloud Template</a></li>
<li><a href="#updating-properties">Updating properties</a>
<ul>
<li><a href="#updating-custom-properties">Updating custom properties</a></li>
<li><a href="#introducting-a-fake-input">Introducting a fake input</a></li>
<li><a href="#updating-the-custom-property-with-the-fake-input">Updating the custom property with the fake input</a></li>
</ul>
</li>
<li><a href="#wrap-up">Wrap-up</a></li>
</ul>
</nav>
</div>
<h2 id="deployment-resources">Deployment resources</h2>
<p>Aria Automation deployment resources can be assigned custom properties. These properties can have an impact on how the resource created (name, size, ...), but can also store arbitrary metadata we want to use later for automation purposes (for example backup policy name).</p>
<p>Sometimes we need to change these values during the lifetime of the resource. For virtual machines the IaaS API provides a convinient way to do that via PATCHing /iaas/api/machines/{id}: we can specify the custom properties and values. The same API is exposed via <a href="https://marketplace.cloud.vmware.com/services/details/vmware-vrealize-orchestrator-plug-in-for-vrealize-automation-8-4-2-1?slug=true">Orchestrator Plug-in for vRealize Automation</a>, one can use
<tt>com.vmware.library.vra.infrastructure.machine/createMachineCustomProperties</tt> action to update the VM's custom properties.</p>
<p>There is no such interface for deployment resources in general, and for custom resource types in particular either. This article discusses the challenges and a solution to update custom properties of deployment resources with custom resource type.</p>
<h2 id="recap-of-the-previous-posts">Recap of the previous posts</h2>
<p>In the previous posts <a href="https://kuklis.github.io/cma/cma/post/vra8-custom-resources-1">Aria Automation Custom Resource Types - The Easy Way</a> and <a href="https://kuklis.github.io/cma/cma/post/vra8-custom-resources-3">Aria Automation Custom Resource Types - Schema update</a> we created, and updated a custom resource type definition. In the third part <a href="https://kuklis.github.io/cma/cma/post/vra8-custom-resources-3">Aria Automation Custom Resource Types - Onboarding</a> we discussed how to get an existing resource under management.</p>
<h2 id="virtual-machine-folder-as-a-custom-resource-type">Virtual Machine folder as a Custom Resource Type</h2>
<p>In this fourth part we well use VC:VmFolder Orchestrator type implemented by the vSphere vCenter Plug-in. Previously we used a Dynamic Type, but we can use any existing vRO inventory object type (provided by plugins) as well as CRT. It does not matter for the current discussion as both work the same way.</p>
<h3 id="create-vm-folder-workflow">Create VM folder workflow</h3>
<p>The inputs of create workflow are: <tt>name</tt> of folder and <tt>owner</tt> (for demonstration purposes). The type of output is a <tt>VC:VmFolder</tt>.</p>
<p>To create a folder, we need a parent folder. For the sake of simplicity we will use a slightly modified version of <tt>com.vmware.library.vc.folder/getRootVmFolder</tt> code, not to fail in case of multiple Datacenters found, always selecting the first one:</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln">1</span><span class="kd">var</span> <span class="nx">vimHosts</span> <span class="o">=</span> <span class="nx">VcPlugin</span><span class="p">.</span><span class="nx">getVimHosts</span><span class="p">();</span>
<span class="ln">2</span><span class="kd">var</span> <span class="nx">rootFolder</span> <span class="o">=</span> <span class="nx">vimHosts</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">rootFolder</span><span class="p">;</span>
<span class="ln">3</span><span class="kd">var</span> <span class="nx">datacenters</span> <span class="o">=</span> <span class="nx">getAllDatacenters</span><span class="p">(</span><span class="nx">rootFolder</span><span class="p">);</span>
<span class="ln">4</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Datacenter: &#34;</span> <span class="o">+</span> <span class="nx">datacenters</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">name</span><span class="p">);</span>
<span class="ln">5</span><span class="nx">rootVMfolder</span> <span class="o">=</span> <span class="nx">datacenters</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">vmFolder</span><span class="p">;</span>
<span class="ln">6</span>
<span class="ln">7</span><span class="kd">function</span> <span class="nx">getAllDatacenters</span><span class="p">(</span><span class="nx">folder</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">8</span><span class="p">...</span>
</code></pre></div><p>After calling the <tt>Library/vCenter/Folder management/VM folder/Create virtual machine folder</tt> to create the folder within the root folder, we set the <tt>owner</tt> Custom Attribute. For that to work, we need to create a new custom attribute in vCenter to store our value:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/new_custom_attribute.png" alt=""><br></p>
<p>The code to store the value, with returning the workflow output object:</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln">1</span><span class="nx">newFolder</span><span class="p">.</span><span class="nx">setCustomValue</span><span class="p">(</span><span class="s2">&#34;owner&#34;</span><span class="p">,</span> <span class="nx">owner</span><span class="p">);</span>
<span class="ln">2</span><span class="nx">folder</span> <span class="o">=</span> <span class="nx">newFolder</span><span class="p">;</span>
</code></pre></div><h3 id="update-vm-folder-delete-vm-folder-workflows">Update VM folder, Delete VM folder workflows</h3>
<p>The update workflow must have an input and output of type VC:VmFolder. We also add the <tt>name</tt> of folder to allow folder renaming.</p>
<p>This workflow is just a wrapper: first we call <tt>Library/vCenter/Folder management/VM folder/Rename virtual machine folder</tt> workflow, and return the VC:VmFolder object.</p>
<p>The delete workflow calls <tt>Library/vCenter/Folder management/VM folder/Delete virtual machine folder</tt> workflow.</p>
<p>With all 3 workflows implemented, we can start defining the CRT.</p>
<h2 id="define-vmfolder-custom-resource-type">Define VMfolder Custom Resource Type</h2>
<p>Refresh the workflow inventory in Cloud Assembly (Start Data Collection):
<img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-2/datacollection.png" alt=""><br></p>
<p>Define the CRT:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/new_crt.png" alt=""><br></p>
<p>The schema is automatically created:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/schema.png" alt=""><br></p>
<p>The input properties <tt>name</tt> and <tt>owner</tt> added from the create workflow inputs, and the output properties <tt>sdkId</tt>, <tt>name</tt> and <tt>id</tt> added from the plugin object type definition of <tt>VC:VmFolder</tt>. This works the same as Dynamic Types.</p>
<p>With that completed, we have a simple CRT to test with.</p>
<h2 id="cloud-template">Cloud Template</h2>
<p>We define three inputs for the template: <tt>name</tt> of the folder, <tt>owner</tt> and a <tt>comment</tt>. Each input is mapped as a property of the resource.</p>
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="ln"> 1</span><span class="nt">formatVersion</span><span class="p">:</span><span class="w"> </span><span class="m">1</span><span class="w">
</span><span class="ln"> 2</span><span class="w"></span><span class="nt">inputs</span><span class="p">:</span><span class="w">
</span><span class="ln"> 3</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w">
</span><span class="ln"> 4</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">string</span><span class="w">
</span><span class="ln"> 5</span><span class="w"> </span><span class="nt">owner</span><span class="p">:</span><span class="w">
</span><span class="ln"> 6</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">string</span><span class="w">
</span><span class="ln"> 7</span><span class="w"> </span><span class="nt">comment</span><span class="p">:</span><span class="w">
</span><span class="ln"> 8</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">string</span><span class="w">
</span><span class="ln"> 9</span><span class="w"></span><span class="nt">resources</span><span class="p">:</span><span class="w">
</span><span class="ln">10</span><span class="w"> </span><span class="nt">Custom_vmfolder</span><span class="p">:</span><span class="w">
</span><span class="ln">11</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Custom.vmfolder</span><span class="w">
</span><span class="ln">12</span><span class="w"> </span><span class="nt">preventDelete</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="ln">13</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">14</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">${input.name}</span><span class="w">
</span><span class="ln">15</span><span class="w"> </span><span class="nt">owner</span><span class="p">:</span><span class="w"> </span><span class="l">${input.owner}</span><span class="w">
</span><span class="ln">16</span><span class="w"> </span><span class="nt">comment</span><span class="p">:</span><span class="w"> </span><span class="l">${input.comment}</span></code></pre></div>
<p>The properties <tt>name</tt> and <tt>owner</tt> are directly used by the create workflow inputs, the <tt>comment</tt> property will be a custom property assigned to the resource. The <tt><a href="https://docs.vmware.com/en/VMware-Aria-Automation/8.17/Using-Automation-Assembler/GUID-B76918AE-D18D-4821-B160-F0CFAE173359.html">preventDelete</a></tt> resource flag prevents folder deletion in case of resource request (update) performed on the object.</p>
<p>This is the new input form when deploying:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/inputs.png" alt=""><br></p>
<p>The created resource:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/resource_created.png" alt=""><br></p>
<p>Note that the <tt>name</tt> and <tt>owner</tt> properties are visible on the GUI, but the <tt>comment</tt> is not. However, we can get it via API:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln"> 1</span>$ curl -sSk -H <span class="s2">&#34;Authorization: Bearer </span><span class="nv">$token</span><span class="s2">&#34;</span> <span class="s1">&#39;https://vra8.corp.local/deployment/api/resources?resourceTypes=Custom.vmfolder&#39;</span><span class="p">|</span>jq .
<span class="ln"> 2</span><span class="o">{</span>
<span class="ln"> 3</span> <span class="s2">&#34;content&#34;</span>: <span class="o">[</span>
<span class="ln"> 4</span> <span class="o">{</span>
<span class="ln"> 5</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;c6b68e9f-bc00-4371-962a-0dfbeefde4c5&#34;</span>,
<span class="ln"> 6</span> <span class="s2">&#34;name&#34;</span>: <span class="s2">&#34;folder1&#34;</span>,
<span class="ln"> 7</span> <span class="s2">&#34;type&#34;</span>: <span class="s2">&#34;Custom.vmfolder&#34;</span>,
<span class="ln"> 8</span> <span class="s2">&#34;properties&#34;</span>: <span class="o">{</span>
<span class="ln"> 9</span> <span class="s2">&#34;owner&#34;</span>: <span class="s2">&#34;kuklis&#34;</span>,
<span class="ln">10</span> <span class="s2">&#34;folder&#34;</span>: <span class="o">{</span>
<span class="ln">11</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;group-v51870&#34;</span>,
<span class="ln">12</span> <span class="s2">&#34;name&#34;</span>: <span class="s2">&#34;folder1&#34;</span>,
<span class="ln">13</span> <span class="s2">&#34;type&#34;</span>: <span class="s2">&#34;VmFolder&#34;</span>,
<span class="ln">14</span> <span class="s2">&#34;@type&#34;</span>: <span class="s2">&#34;VmFolder&#34;</span>,
<span class="ln">15</span> <span class="s2">&#34;sdkId&#34;</span>: <span class="s2">&#34;vc.corp.local/group-v51870&#34;</span>,
<span class="ln">16</span> <span class="s2">&#34;dunesId&#34;</span>: <span class="s2">&#34;vc.corp.local,id:group-v51870&#34;</span>,
<span class="ln">17</span> <span class="s2">&#34;@fullType&#34;</span>: <span class="s2">&#34;VC:VmFolder&#34;</span>
<span class="ln">18</span> <span class="o">}</span>,
<span class="ln">19</span> <span class="s2">&#34;name&#34;</span>: <span class="s2">&#34;folder1&#34;</span>,
<span class="ln">20</span> <span class="s2">&#34;resourceName&#34;</span>: <span class="s2">&#34;folder1&#34;</span>,
<span class="hl"><span class="ln">21</span> <span class="s2">&#34;comment&#34;</span>: <span class="s2">&#34;comment1&#34;</span>,
</span><span class="ln">22</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;a8caa348-7980-4da9-b4ee-c8f9647b619c&#34;</span>
<span class="ln">23</span> <span class="o">}</span>,
<span class="ln">24</span> <span class="s2">&#34;createdAt&#34;</span>: <span class="s2">&#34;2024-07-03T18:28:02.245705Z&#34;</span>,
<span class="ln">25</span> <span class="s2">&#34;syncStatus&#34;</span>: <span class="s2">&#34;SUCCESS&#34;</span>,
<span class="ln">26</span> <span class="s2">&#34;origin&#34;</span>: <span class="s2">&#34;DEPLOYED&#34;</span>,
<span class="ln">27</span> <span class="s2">&#34;deploymentId&#34;</span>: <span class="s2">&#34;6341b47c-0f1d-4330-82ac-1eecfa0f8d87&#34;</span>,
<span class="ln">28</span> <span class="s2">&#34;projectId&#34;</span>: <span class="s2">&#34;ffab9ffd-a072-4617-bbb4-54101b3b5dd1&#34;</span>,
<span class="ln">29</span> <span class="s2">&#34;orgId&#34;</span>: <span class="s2">&#34;360bc331-1360-4d76-8000-aae9f7491989&#34;</span>,
<span class="ln">30</span> <span class="s2">&#34;billable&#34;</span>: <span class="nb">false</span>
<span class="ln">31</span> <span class="o">}</span>
<span class="ln">32</span> <span class="o">]</span>,
<span class="ln">33</span>...</code></pre></div>
<p>The resulting VM folder on vCenter:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/vmfolder.png" alt=""></p>
<h2 id="updating-properties">Updating properties</h2>
<p>Custom properties are directly mapped to deployment inputs. We can change them by updating (changing) the inputs.</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/update_menu.png" alt=""><br></p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/update_inputs.png" alt=""><br></p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/update_plan.png" alt=""><br></p>
<p>The updated deployment resource has updated custom properties (top) and resource properties (bottom):</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/resource_updated.png" alt=""><br></p>
<p>The VM folder is renamed:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/vmfolder2.png" alt=""><br></p>
<h3 id="updating-custom-properties">Updating custom properties</h3>
<p>Let's try to update a custom property, the <tt>comment</tt> field:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/update_comment.png" alt=""><br></p>
<p>We see that Aria Automation detects the resource does not need to be updated, as the input has no effect on the object (not mapped to an input):</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/update_plan2.png" alt=""><br></p>
<p>If we submit the update request, and check the <tt>comment</tt> custom property, we see no change:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln"> 1</span>curl -sSk -H <span class="s2">&#34;Authorization: Bearer </span><span class="nv">$token</span><span class="s2">&#34;</span> <span class="s1">&#39;https://vra8.corp.local/deployment/api/resources?resourceTypes=Custom.vmfolder&#39;</span><span class="p">|</span>jq .
<span class="ln"> 2</span><span class="o">{</span>
<span class="ln"> 3</span> <span class="s2">&#34;content&#34;</span>: <span class="o">[</span>
<span class="ln"> 4</span> <span class="o">{</span>
<span class="ln"> 5</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;c6b68e9f-bc00-4371-962a-0dfbeefde4c5&#34;</span>,
<span class="ln"> 6</span> <span class="s2">&#34;name&#34;</span>: <span class="s2">&#34;folder2&#34;</span>,
<span class="ln"> 7</span> <span class="s2">&#34;type&#34;</span>: <span class="s2">&#34;Custom.vmfolder&#34;</span>,
<span class="ln"> 8</span> <span class="s2">&#34;properties&#34;</span>: <span class="o">{</span>
<span class="ln"> 9</span> <span class="s2">&#34;owner&#34;</span>: <span class="s2">&#34;kuklis&#34;</span>,
<span class="ln">10</span> <span class="s2">&#34;folder&#34;</span>: <span class="o">{</span>
<span class="ln">11</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;group-v51870&#34;</span>,
<span class="ln">12</span> <span class="s2">&#34;name&#34;</span>: <span class="s2">&#34;folder2&#34;</span>,
<span class="ln">13</span> <span class="s2">&#34;type&#34;</span>: <span class="s2">&#34;VmFolder&#34;</span>,
<span class="ln">14</span> <span class="s2">&#34;@type&#34;</span>: <span class="s2">&#34;VmFolder&#34;</span>,
<span class="ln">15</span> <span class="s2">&#34;sdkId&#34;</span>: <span class="s2">&#34;vc.corp.local/group-v51870&#34;</span>,
<span class="ln">16</span> <span class="s2">&#34;dunesId&#34;</span>: <span class="s2">&#34;vc.corp.local,id:group-v51870&#34;</span>,
<span class="ln">17</span> <span class="s2">&#34;@fullType&#34;</span>: <span class="s2">&#34;VC:VmFolder&#34;</span>
<span class="ln">18</span> <span class="o">}</span>,
<span class="ln">19</span> <span class="s2">&#34;folder_in&#34;</span>: <span class="o">{</span>
<span class="ln">20</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;vc2.poc.local,id:group-v51870&#34;</span>,
<span class="ln">21</span> <span class="s2">&#34;type&#34;</span>: <span class="s2">&#34;VC:VmFolder&#34;</span>
<span class="ln">22</span> <span class="o">}</span>,
<span class="ln">23</span> <span class="s2">&#34;name&#34;</span>: <span class="s2">&#34;folder2&#34;</span>,
<span class="ln">24</span> <span class="s2">&#34;resourceName&#34;</span>: <span class="s2">&#34;folder2&#34;</span>,
<span class="hl"><span class="ln">25</span> <span class="s2">&#34;comment&#34;</span>: <span class="s2">&#34;comment1&#34;</span>,
</span><span class="ln">26</span> <span class="s2">&#34;id&#34;</span>: <span class="s2">&#34;a8caa348-7980-4da9-b4ee-c8f9647b619c&#34;</span>
<span class="ln">27</span> <span class="o">}</span>,
<span class="ln">28</span> <span class="s2">&#34;createdAt&#34;</span>: <span class="s2">&#34;2024-07-03T18:28:02.245705Z&#34;</span>,
<span class="ln">29</span> <span class="s2">&#34;syncStatus&#34;</span>: <span class="s2">&#34;SUCCESS&#34;</span>,
<span class="ln">30</span> <span class="s2">&#34;origin&#34;</span>: <span class="s2">&#34;DEPLOYED&#34;</span>,
<span class="ln">31</span> <span class="s2">&#34;deploymentId&#34;</span>: <span class="s2">&#34;6341b47c-0f1d-4330-82ac-1eecfa0f8d87&#34;</span>,
<span class="ln">32</span> <span class="s2">&#34;projectId&#34;</span>: <span class="s2">&#34;ffab9ffd-a072-4617-bbb4-54101b3b5dd1&#34;</span>,
<span class="ln">33</span> <span class="s2">&#34;orgId&#34;</span>: <span class="s2">&#34;360bc331-1360-4d76-8000-aae9f7491989&#34;</span>,
<span class="ln">34</span> <span class="s2">&#34;billable&#34;</span>: <span class="nb">false</span>
<span class="ln">35</span> <span class="o">}</span>
<span class="ln">36</span> <span class="o">]</span>,
<span class="ln">37</span>...</code></pre></div>
<p>In order to update a resource property, we need to add the property as an input to the <em>create</em> and <em>update</em> workflows to trigger a deployment resource update.</p>
<h3 id="introducting-a-fake-input">Introducting a fake input</h3>
<p>This will have no direct affect on the VM folder creation or update process, but will trick the automation engine to run the update workflow and update the custom property a the same time.</p>
<p>We need to refresh the workflow inventory in Cloud Assembly (Start Data Collection) again, and update the custom resource type schema:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/schema_update.png" alt=""><br></p>
<h3 id="updating-the-custom-property-with-the-fake-input">Updating the custom property with the fake input</h3>
<p>Let's try to update the <tt>comment</tt> custom field now. It triggers a deployment resource update and changes the comment property:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-4/comment_updated.png" alt=""></p>
<p>Note that the property is visible on the GUI now.</p>
<h2 id="wrap-up">Wrap-up</h2>
<p>In order to update a custom property, we also need to change any property that is an input of the <em>create</em> and <em>update</em> workflows. This can be the property itself, by adding it as an input to these workflows (as in the example above).</p>
<p>But if we want to keep it hidden off the GUI, we can introduce another fake input, for example <tt>updateVersion</tt>. In this case, by changing both the (hidden) property and the <tt>updateVersion</tt> inputs, we can force a resource update and the custom property value updated at the same time.</p>
<p>You can find the sample workflows at GitHub: <a href="https://github.com/kuklis/vro8-packages">https://github.com/kuklis/vro8-packages</a><br>
com.dynamictypes.vmfolder.package includes the original workflows, and com.dynamictypes.vmfolder2.package the updated versions.</p>
</description>
</item>
<item>
<title>Aria Automation Custom Resource Types - Onboarding</title>
<link>https://kuklis.github.io/cma/post/vra8-custom-resources-3/</link>
<pubDate>Tue, 12 Mar 2024 00:00:00 +0000</pubDate>
<guid>https://kuklis.github.io/cma/post/vra8-custom-resources-3/</guid>
<description>
<p>How to onboard existing instances of a custom resource type</p>
<div class="toc">
<H3>Table of Contents</H3>
<nav id="TableOfContents">
<ul>
<li><a href="#why-we-need-onboarding">Why we need onboarding</a></li>
<li><a href="#recap-of-the-previous-posts">Recap of the previous posts</a></li>
<li><a href="#how-to-onboard">How to onboard</a>
<ul>
<li><a href="#create-myobject-workflow-modifications">Create myObject workflow modifications</a></li>
<li><a href="#update-the-custom-resource-type-and-the-cloud-template">Update the custom resource type and the cloud template</a></li>
</ul>
</li>
<li><a href="#how-to-offboard">How to offboard</a>
<ul>
<li><a href="#destroy-myobject-workflow-modifications">Destroy myObject workflow modifications</a></li>
<li><a href="#deleting-a-deployment">Deleting a deployment</a></li>
</ul>
</li>
<li><a href="#wrap-up">Wrap-up</a></li>
</ul>
</nav>
</div>
<h2 id="why-we-need-onboarding">Why we need onboarding</h2>
<p>Often there are existing real world objects at the time of introducing Aria Automation custom resource types. These brown field scenarios require a way to onboard existing instances of the resource without recreating them. This post will present a way to do that.</p>
<h2 id="recap-of-the-previous-posts">Recap of the previous posts</h2>
<p>In the previous posts <a href="https://kuklis.github.io/cma/cma/post/vra8-custom-resources-1">Aria Automation Custom Resource Types - The Easy Way</a> and <a href="https://kuklis.github.io/cma/cma/post/vra8-custom-resources-3">Aria Automation Custom Resource Types - Schema update</a> we created and updated a custom resource type definition.</p>
<p>In the third part we assume the steps in the previous post are completed, we have the Dynamic Type, the CRT and the CT created.</p>
<h2 id="how-to-onboard">How to onboard</h2>
<p>For the sample myObject type, let's add a new input called <tt>existing</tt> meaning we do not want to create a new object but rather reuse an existing one.</p>
<h3 id="create-myobject-workflow-modifications">Create myObject workflow modifications</h3>
<p>We need to introduce a new input <tt>&quot;existing&quot;</tt> to the workflow, and use it to find the existing object, instead of creating it. In our case this will be an SQL SELECT instead of an SQL INSERT:</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln"> 1</span><span class="kd">var</span> <span class="nx">id</span><span class="p">;</span>
<span class="ln"> 2</span><span class="k">if</span> <span class="p">(</span><span class="nx">existing</span><span class="p">)</span> <span class="p">{</span>
<span class="ln"> 3</span> <span class="kd">var</span> <span class="nx">records</span> <span class="o">=</span> <span class="nx">db</span><span class="p">.</span><span class="nx">readCustomQuery</span><span class="p">(</span><span class="s2">&#34;SELECT id,name FROM myObjects WHERE name=&#39;&#34;</span> <span class="o">+</span> <span class="nx">name</span> <span class="o">+</span> <span class="s2">&#34;&#39;&#34;</span><span class="p">);</span>
<span class="ln"> 4</span> <span class="k">if</span> <span class="p">(</span><span class="nx">records</span><span class="p">.</span><span class="nx">length</span><span class="p">)</span> <span class="nx">id</span> <span class="o">=</span> <span class="nx">records</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">getProperty</span><span class="p">(</span><span class="s2">&#34;id&#34;</span><span class="p">);</span>
<span class="ln"> 5</span> <span class="k">else</span> <span class="k">throw</span> <span class="s2">&#34;Object with name &#39;&#34;</span> <span class="o">+</span> <span class="nx">name</span> <span class="o">+</span> <span class="s2">&#34;&#39; not found&#34;</span><span class="p">;</span>
<span class="ln"> 6</span><span class="p">}</span>
<span class="ln"> 7</span><span class="k">else</span> <span class="p">{</span>
<span class="ln"> 8</span> <span class="nx">id</span> <span class="o">=</span> <span class="nx">System</span><span class="p">.</span><span class="nx">nextUUID</span><span class="p">();</span>
<span class="ln"> 9</span> <span class="kd">var</span> <span class="nx">date</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">();</span>
<span class="ln">10</span>
<span class="ln">11</span> <span class="nx">db</span><span class="p">.</span><span class="nx">executeCustomQuery</span><span class="p">(</span><span class="s2">&#34;INSERT INTO myObjects (id,name,weight,created) VALUES &#34;</span> <span class="o">+</span>
<span class="ln">12</span> <span class="s2">&#34;(&#39;&#34;</span> <span class="o">+</span> <span class="nx">id</span> <span class="o">+</span> <span class="s2">&#34;&#39;,&#39;&#34;</span> <span class="o">+</span> <span class="nx">name</span> <span class="o">+</span> <span class="s2">&#34; - (&#34;</span> <span class="o">+</span> <span class="nx">comment</span> <span class="o">+</span> <span class="s2">&#34;)&#39;,&#34;</span> <span class="o">+</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">round</span><span class="p">(</span><span class="nx">weight</span><span class="p">)</span> <span class="o">+</span> <span class="s2">&#34;,&#39;&#34;</span> <span class="o">+</span> <span class="nx">date</span><span class="p">.</span><span class="nx">toISOString</span><span class="p">()</span> <span class="o">+</span> <span class="s2">&#34;&#39;)&#34;</span><span class="p">);</span>
<span class="ln">13</span><span class="p">}</span>
<span class="ln">14</span><span class="nx">myObject</span> <span class="o">=</span> <span class="nx">DynamicTypesManager</span><span class="p">.</span><span class="nx">getObject</span><span class="p">(</span><span class="s2">&#34;myNS&#34;</span><span class="p">,</span> <span class="s2">&#34;myObject&#34;</span><span class="p">,</span> <span class="nx">id</span><span class="p">);</span> <span class="c1">// call Find by Id
</span><span class="ln">15</span><span class="c1"></span>
<span class="ln">16</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">myObject</span><span class="p">.</span><span class="nx">toSource</span><span class="p">());</span>
</code></pre></div><h3 id="update-the-custom-resource-type-and-the-cloud-template">Update the custom resource type and the cloud template</h3>
<p>Refresh the workflow inventory in Cloud Assembly (Start Data Collection):
<img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-2/datacollection.png" alt=""><br></p>
<p>and modify the custom resource schema: add the new property:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/update.png" alt=""><br></p>
<p>Add a new input into the cloud template and map it to the new property:</p>
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="ln"> 1</span><span class="nt">formatVersion</span><span class="p">:</span><span class="w"> </span><span class="m">1</span><span class="w">
</span><span class="ln"> 2</span><span class="w"></span><span class="nt">inputs</span><span class="p">:</span><span class="w">
</span><span class="ln"> 3</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w">
</span><span class="ln"> 4</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">string</span><span class="w">
</span><span class="ln"> 5</span><span class="w"> </span><span class="nt">weight</span><span class="p">:</span><span class="w">
</span><span class="ln"> 6</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">integer</span><span class="w">
</span><span class="ln"> 7</span><span class="w"> </span><span class="nt">comment</span><span class="p">:</span><span class="w">
</span><span class="ln"> 8</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">string</span><span class="w">
</span><span class="hl"><span class="ln"> 9</span><span class="w"> </span><span class="nt">existing</span><span class="p">:</span><span class="w">
</span></span><span class="hl"><span class="ln">10</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">boolean</span><span class="w">
</span></span><span class="ln">11</span><span class="w"></span><span class="nt">resources</span><span class="p">:</span><span class="w">
</span><span class="ln">12</span><span class="w"> </span><span class="nt">myObject_1</span><span class="p">:</span><span class="w">
</span><span class="ln">13</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Custom.myObject</span><span class="w">
</span><span class="ln">14</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">15</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">${input.name}</span><span class="w">
</span><span class="ln">16</span><span class="w"> </span><span class="nt">weight</span><span class="p">:</span><span class="w"> </span><span class="l">${input.weight}</span><span class="w">
</span><span class="ln">17</span><span class="w"> </span><span class="nt">comment</span><span class="p">:</span><span class="w"> </span><span class="l">${input.comment}</span><span class="w">
</span><span class="hl"><span class="ln">18</span><span class="w"> </span><span class="nt">existing</span><span class="p">:</span><span class="w"> </span><span class="l">${input.existing}</span></span></code></pre></div>
<p>This is the new input form when deploying:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/inputs.png" alt=""><br></p>
<p>Our table with the pre-existing record we want to onboard:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/table.png" alt=""><br></p>
<p>and the deployment resource:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/myObject.png" alt=""><br></p>
<p>Id and create timestamp are fetched from the database table.</p>
<h2 id="how-to-offboard">How to offboard</h2>
<p>There are no input parameters for deleting a deployment or deployment resource, so we need to find another way to prevent deleting an object from the database. There are many options, like introduction an extra column with &quot;protected&quot; flag, or use some naming scheme to distinguish the objects we want to keep. In our example we choose a simple method: if the comment is <tt>(keep)</tt>, the custom resource delete workflow will not remove the database entry.</p>
<h3 id="destroy-myobject-workflow-modifications">Destroy myObject workflow modifications</h3>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln">1</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Destroying: &#34;</span> <span class="o">+</span> <span class="nx">input</span><span class="p">.</span><span class="nx">toSource</span><span class="p">());</span>
<span class="ln">2</span><span class="k">if</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span> <span class="o">!=</span> <span class="nx">input</span><span class="p">.</span><span class="nx">name</span><span class="p">.</span><span class="nx">indexOf</span><span class="p">(</span><span class="s2">&#34;(keep)&#34;</span><span class="p">))</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Keeping &#34;</span> <span class="o">+</span> <span class="nx">input</span><span class="p">.</span><span class="nx">name</span><span class="p">);</span>
<span class="ln">3</span><span class="k">else</span> <span class="nx">db</span><span class="p">.</span><span class="nx">executeCustomQuery</span><span class="p">(</span><span class="s2">&#34;DELETE FROM myObjects WHERE id=&#39;&#34;</span> <span class="o">+</span> <span class="nx">input</span><span class="p">.</span><span class="nx">id</span> <span class="o">+</span> <span class="s2">&#34;&#39;&#34;</span><span class="p">);</span>
<span class="ln">4</span><span class="nx">actionResult</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
</code></pre></div><h3 id="deleting-a-deployment">Deleting a deployment</h3>
<p>Let's delete a deployment with <tt>(keep)</tt> comment of the resource:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/delete.png" alt=""><br></p>
<p>The workflow logs:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/logs.png" alt=""><br></p>
<p>The object still exists on the backend (in the DB table, in our case):</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-custom-resources-3/table.png" alt=""><br></p>
<h2 id="wrap-up">Wrap-up</h2>
<p>We may need to onboard or offboard custom resource type objects. The above method allows us to do that without modifying the &quot;real world&quot; instances.</p>
<p>You can find the sample workflows at GitHub: <a href="https://github.com/kuklis/vro8-packages">https://github.com/kuklis/vro8-packages</a><br>
com.dynamictypes.myobject-3.package includes the modifications.</p>
</description>
</item>
<item>
<title>Aria Automation IPAM Integration - the Easy Way</title>
<link>https://kuklis.github.io/cma/post/vra8-ipam-in-vro/</link>
<pubDate>Wed, 13 Dec 2023 00:00:00 +0000</pubDate>
<guid>https://kuklis.github.io/cma/post/vra8-ipam-in-vro/</guid>
<description>
<p>How to implement external IPAM integration via Orchestrator</p>
<div class="toc">
<H3>Table of Contents</H3>
<nav id="TableOfContents">
<ul>
<li><a href="#motivation">Motivation</a></li>
<li><a href="#the-cloud-template">The Cloud Template</a></li>
<li><a href="#network-profiles">Network Profiles</a></li>
<li><a href="#event-broker-subscription">Event Broker Subscription</a></li>
<li><a href="#event-data">Event Data</a></li>
<li><a href="#matching-networks-against-ipam">Matching networks against IPAM</a></li>
<li><a href="#phpipam-functions">phpIPAM Functions</a></li>
<li><a href="#expected-workflow-output">Expected workflow output</a></li>
<li><a href="#putting-ip-allocate-code-all-together">Putting IP Allocate code all together</a></li>
<li><a href="#ip-release">IP Release</a></li>
<li><a href="#conclusion">Conclusion</a></li>
</ul>
</nav>
</div>
<h2 id="motivation">Motivation</h2>
<p>vRealize Automation version 7.x external IPAM support was provided by Orchestrator actions/workflows that allowed an easy way to integrate external providers to the IaaS service. Aria Automation version 8.x changed this model, as it is a SAAS first application and the cloud version does not have Orchestrator. So the engineers came up with a container-based solution called <em>VMware Aria Automation Third-Party IPAM SDK</em>. This skeleton code allows to create custom IPAM providers running as extensibility actions (ABX) within containers. This approach is very different from what we had in v7.x and made the development/debugging more complex. We took the challenge and implemented a phpIPAM provider by the SDK. I cannot share the result but if you are interested there are some public implementations, e.g. <a href="https://github.com/jbowdre/phpIPAM-for-vRA8">https://github.com/jbowdre/phpIPAM-for-vRA8</a>.</p>
<p>Having an on-premises Aria Automation (with Orchestrator) I wondered if the integration can implemented via vRO workflows. This would allow easier development process, more (existing) Orchestator integrations and better visibility of code runs. In the following chapters we'll reimplement our phpIPAM plugin functionality in pure Orchestrator JavaScript code.</p>
<h2 id="the-cloud-template">The Cloud Template</h2>
<p>Let's create a cloud template that has multiple virtual machines, with multiple interfaces to make sure our code would work with any possible combination.</p>
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="ln"> 1</span><span class="nt">formatVersion</span><span class="p">:</span><span class="w"> </span><span class="m">1</span><span class="w">
</span><span class="ln"> 2</span><span class="w"></span><span class="nt">inputs</span><span class="p">:</span><span class="w">
</span><span class="ln"> 3</span><span class="w"> </span><span class="nt">count</span><span class="p">:</span><span class="w">
</span><span class="ln"> 4</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">integer</span><span class="w">
</span><span class="ln"> 5</span><span class="w"> </span><span class="nt">default</span><span class="p">:</span><span class="w"> </span><span class="m">1</span><span class="w">
</span><span class="ln"> 6</span><span class="w"></span><span class="nt">resources</span><span class="p">:</span><span class="w">
</span><span class="ln"> 7</span><span class="w"> </span><span class="nt">internal</span><span class="p">:</span><span class="w">
</span><span class="ln"> 8</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Cloud.vSphere.Machine</span><span class="w">
</span><span class="ln"> 9</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">10</span><span class="w"> </span><span class="nt">count</span><span class="p">:</span><span class="w"> </span><span class="l">${input.count}</span><span class="w">
</span><span class="ln">11</span><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">photon</span><span class="w">
</span><span class="ln">12</span><span class="w"> </span><span class="nt">flavor</span><span class="p">:</span><span class="w"> </span><span class="l">small</span><span class="w">
</span><span class="ln">13</span><span class="w"> </span><span class="nt">networks</span><span class="p">:</span><span class="w">
</span><span class="ln">14</span><span class="w"> </span>- <span class="nt">network</span><span class="p">:</span><span class="w"> </span><span class="l">${resource.lan0.id}</span><span class="w">
</span><span class="ln">15</span><span class="w"> </span>- <span class="nt">network</span><span class="p">:</span><span class="w"> </span><span class="l">${resource.lan1.id}</span><span class="w">
</span><span class="ln">16</span><span class="w"> </span><span class="nt">public</span><span class="p">:</span><span class="w">
</span><span class="ln">17</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Cloud.vSphere.Machine</span><span class="w">
</span><span class="ln">18</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">19</span><span class="w"> </span><span class="nt">count</span><span class="p">:</span><span class="w"> </span><span class="l">${input.count}</span><span class="w">
</span><span class="ln">20</span><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">photon</span><span class="w">
</span><span class="ln">21</span><span class="w"> </span><span class="nt">flavor</span><span class="p">:</span><span class="w"> </span><span class="l">small</span><span class="w">
</span><span class="ln">22</span><span class="w"> </span><span class="nt">networks</span><span class="p">:</span><span class="w">
</span><span class="ln">23</span><span class="w"> </span>- <span class="nt">network</span><span class="p">:</span><span class="w"> </span><span class="l">${resource.dmz.id}</span><span class="w">
</span><span class="ln">24</span><span class="w"> </span><span class="nt">lan0</span><span class="p">:</span><span class="w">
</span><span class="ln">25</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Cloud.vSphere.Network</span><span class="w">
</span><span class="ln">26</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">27</span><span class="w"> </span><span class="nt">networkType</span><span class="p">:</span><span class="w"> </span><span class="l">existing</span><span class="w">
</span><span class="ln">28</span><span class="w"> </span><span class="nt">constraints</span><span class="p">:</span><span class="w">
</span><span class="ln">29</span><span class="w"> </span>- <span class="nt">tag</span><span class="p">:</span><span class="w"> </span><span class="l">zone:lan</span><span class="w">
</span><span class="ln">30</span><span class="w"> </span><span class="nt">lan1</span><span class="p">:</span><span class="w">
</span><span class="ln">31</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Cloud.vSphere.Network</span><span class="w">
</span><span class="ln">32</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">33</span><span class="w"> </span><span class="nt">networkType</span><span class="p">:</span><span class="w"> </span><span class="l">existing</span><span class="w">
</span><span class="ln">34</span><span class="w"> </span><span class="nt">constraints</span><span class="p">:</span><span class="w">
</span><span class="ln">35</span><span class="w"> </span>- <span class="nt">tag</span><span class="p">:</span><span class="w"> </span><span class="l">zone:mgt</span><span class="w">
</span><span class="ln">36</span><span class="w"> </span><span class="nt">dmz</span><span class="p">:</span><span class="w">
</span><span class="ln">37</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l">Cloud.vSphere.Network</span><span class="w">
</span><span class="ln">38</span><span class="w"> </span><span class="nt">properties</span><span class="p">:</span><span class="w">
</span><span class="ln">39</span><span class="w"> </span><span class="nt">networkType</span><span class="p">:</span><span class="w"> </span><span class="l">existing</span><span class="w">
</span><span class="ln">40</span><span class="w"> </span><span class="nt">constraints</span><span class="p">:</span><span class="w">
</span><span class="ln">41</span><span class="w"> </span>- <span class="nt">tag</span><span class="p">:</span><span class="w"> </span><span class="l">zone:dmz</span><span class="w">
</span><span class="ln">42</span><span class="w">
</span></code></pre></div><p><img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/ct.png" alt=""></p>
<p>The <tt>internal</tt> VMs connect to the <em>lan</em> and <em>mgt</em> zones, while the <tt>public</tt> VMs connect to the <em>dmz</em> zone.</p>
<h2 id="network-profiles">Network Profiles</h2>
<p>We create 3 profiles. The <tt>Prod</tt> profile will cover the network connections of the internal VMs, while <tt>DMZ1</tt> and <tt>DMZ2</tt> profiles can provide networks for the public VMs. It is important to note that each VM with the same connection combination must use a single network profile, containing all the matching networks required to connect to.<br>
<img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/profile_prod.png" alt=""><br>
<img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/profile_dmz1.png" alt=""><br>
<img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/profile_dmz2.png" alt=""><br></p>
<p>About the profile selection process: the Provisioning Service can select <tt>Prod</tt> profile for the internal VMs, and can select from <tt>DMZ1</tt> and <tt>DMZ2</tt> for public VMs. Within <tt>Prod</tt> profile there are two networks that can connect to <em>lan</em> zone: net1-pr-nsxt-ls and net2-pp-nsxt-ls. <em>mgt</em> zone is only available via net3-mg-nsxt-ls. Also, within the DMZ profiles, only a single network can be selected. Let's see how these options are presented.</p>
<h2 id="event-broker-subscription">Event Broker Subscription</h2>
<p>EBS is the super glue of Aria Automation: it allows us to subscribe to a particular event and run arbitrary code once it is triggered. In order to create IPAM integration and network selections for our IaaS cloud template, we need to use the <em>Network Configure</em> (network.configure) event topic for VM creation and <em>Compute Post Removal</em> (compute.removal.post) event topic for VM deletion. Both topics require workflows with the usual <tt>inputProperties</tt> input. The following outputs are available for <tt>Network Configure</tt>:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/ebs_output.png" alt=""><br></p>
<p>Important! Make sure to create the EBS with <em>blocking</em> settings, otherwise the return values are not used in the deployment process.</p>
<p>Many attributes of network configuration can be overriden in this deployment phase. For the sake of simplicity we will set only <tt>networkSelectionIds</tt> and <tt>addresses</tt>, the others will be configured on the network level (CIDR, gateways, DNS servers, etc.). They could also be provided by the IPAM solution, though.</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/network_settings.png" alt=""><br></p>
<h2 id="event-data">Event Data</h2>
<p>Let's create a workflow that is triggered at <tt>Network Configure</tt> topic with the <tt>inputProperties</tt> input and print out the input variable value to see what we need to process:</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln">1</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">inputProperties</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">));</span>
</code></pre></div><p>When we request 2x2 VMs (count = 2) we'll see 2 runs of the workflow. The output of the first run:</p>
<div class="highlight"><pre class="chroma"><code class="language-json" data-lang="json"><span class="ln"> 1</span><span class="p">{</span>
<span class="ln"> 2</span> <span class="nt">&#34;componentId&#34;</span><span class="p">:</span> <span class="s2">&#34;internal&#34;</span><span class="p">,</span>
<span class="ln"> 3</span> <span class="nt">&#34;endpointId&#34;</span><span class="p">:</span> <span class="s2">&#34;d098094c-5e88-4428-bcca-83e5ab6e9f6b&#34;</span><span class="p">,</span>
<span class="ln"> 4</span> <span class="nt">&#34;externalIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln"> 5</span> <span class="s2">&#34;internal-342&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">first</span> <span class="err">VM</span> <span class="err">hostname</span>
<span class="ln"> 6</span> <span class="s2">&#34;internal-343&#34;</span> <span class="err">//</span> <span class="err">second</span> <span class="err">VM</span> <span class="err">hostname</span>
<span class="ln"> 7</span> <span class="p">],</span>
<span class="ln"> 8</span> <span class="nt">&#34;blueprintId&#34;</span><span class="p">:</span> <span class="s2">&#34;d2ce10f3-6671-4588-9bf7-9d4b47085865&#34;</span><span class="p">,</span>
<span class="ln"> 9</span> <span class="nt">&#34;tags&#34;</span><span class="p">:</span> <span class="p">{</span>
<span class="ln">10</span> <span class="nt">&#34;project&#34;</span><span class="p">:</span> <span class="s2">&#34;Prod1&#34;</span>
<span class="ln">11</span> <span class="p">},</span>
<span class="ln">12</span> <span class="nt">&#34;customProperties&#34;</span><span class="p">:</span> <span class="p">{</span>
<span class="ln">13</span> <span class="nt">&#34;flavor&#34;</span><span class="p">:</span> <span class="s2">&#34;small&#34;</span><span class="p">,</span>
<span class="ln">14</span> <span class="nt">&#34;neglectPowerOffVms&#34;</span><span class="p">:</span> <span class="s2">&#34;false&#34;</span><span class="p">,</span>
<span class="ln">15</span> <span class="nt">&#34;image&#34;</span><span class="p">:</span> <span class="s2">&#34;photon&#34;</span><span class="p">,</span>
<span class="ln">16</span> <span class="nt">&#34;zone_overlapping_migrated&#34;</span><span class="p">:</span> <span class="s2">&#34;true&#34;</span><span class="p">,</span>
<span class="ln">17</span> <span class="nt">&#34;count&#34;</span><span class="p">:</span> <span class="s2">&#34;2&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">two</span> <span class="err">VMs</span> <span class="err">requested</span>
<span class="ln">18</span> <span class="nt">&#34;project&#34;</span><span class="p">:</span> <span class="s2">&#34;ffab9ffd-a072-4617-bbb4-54101b3b5dd1&#34;</span><span class="p">,</span>
<span class="ln">19</span> <span class="nt">&#34;flavorMappingName&#34;</span><span class="p">:</span> <span class="s2">&#34;small&#34;</span><span class="p">,</span>
<span class="ln">20</span> <span class="nt">&#34;isSimulate&#34;</span><span class="p">:</span> <span class="s2">&#34;false&#34;</span>
<span class="ln">21</span> <span class="p">},</span>
<span class="ln">22</span> <span class="nt">&#34;networkProfileIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">23</span> <span class="s2">&#34;3aca5c1f-443f-42fa-9b4b-66e853ad11f3&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">first</span> <span class="err">VM</span> <span class="err">selected</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">24</span> <span class="s2">&#34;3aca5c1f-443f-42fa-9b4b-66e853ad11f3&#34;</span> <span class="err">//</span> <span class="err">second</span> <span class="err">VM</span> <span class="err">selected</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">25</span> <span class="p">],</span>
<span class="ln">26</span> <span class="nt">&#34;componentTypeId&#34;</span><span class="p">:</span> <span class="s2">&#34;Cloud.vSphere.Machine&#34;</span><span class="p">,</span>
<span class="ln">27</span> <span class="nt">&#34;requestId&#34;</span><span class="p">:</span> <span class="s2">&#34;3bdde4cf-3b12-4b43-90d8-7ccf2c47b1b6&#34;</span><span class="p">,</span>
<span class="ln">28</span> <span class="nt">&#34;deploymentId&#34;</span><span class="p">:</span> <span class="s2">&#34;fa4b9df4-9664-474a-a3f2-056db527e6e3&#34;</span><span class="p">,</span>
<span class="ln">29</span> <span class="nt">&#34;zoneId&#34;</span><span class="p">:</span> <span class="s2">&#34;d8666ddd-804e-4587-a37e-58b72708743d&#34;</span><span class="p">,</span>
<span class="ln">30</span> <span class="nt">&#34;networkSelectionIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">31</span> <span class="p">[</span> <span class="err">//</span> <span class="err">first</span> <span class="err">VM</span>
<span class="ln">32</span> <span class="p">[</span> <span class="err">//</span> <span class="err">first</span> <span class="err">NIC</span>
<span class="ln">33</span> <span class="s2">&#34;c2168e8f-62aa-45e2-9dd9-502d22359214&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">34</span> <span class="s2">&#34;6bf0d8fb-6f39-4a43-b5d9-56a371030171&#34;</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">35</span> <span class="p">],</span>
<span class="ln">36</span> <span class="p">[</span> <span class="err">//</span> <span class="err">second</span> <span class="err">NIC</span>
<span class="ln">37</span> <span class="s2">&#34;dd65009b-3900-46ee-a6a2-d3226d755f4b&#34;</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">38</span> <span class="p">]</span>
<span class="ln">39</span> <span class="p">],</span>
<span class="ln">40</span> <span class="p">[</span> <span class="err">//</span> <span class="err">second</span> <span class="err">VM</span>
<span class="ln">41</span> <span class="p">[</span> <span class="err">//</span> <span class="err">first</span> <span class="err">NIC</span>
<span class="ln">42</span> <span class="s2">&#34;c2168e8f-62aa-45e2-9dd9-502d22359214&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">43</span> <span class="s2">&#34;6bf0d8fb-6f39-4a43-b5d9-56a371030171&#34;</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">44</span> <span class="p">],</span>
<span class="ln">45</span> <span class="p">[</span> <span class="err">//</span> <span class="err">second</span> <span class="err">NIC</span>
<span class="ln">46</span> <span class="s2">&#34;dd65009b-3900-46ee-a6a2-d3226d755f4b&#34;</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">47</span> <span class="p">]</span>
<span class="ln">48</span> <span class="p">]</span>
<span class="ln">49</span> <span class="p">],</span>
<span class="ln">50</span> <span class="nt">&#34;projectId&#34;</span><span class="p">:</span> <span class="s2">&#34;ffab9ffd-a072-4617-bbb4-54101b3b5dd1&#34;</span><span class="p">,</span>
<span class="ln">51</span> <span class="nt">&#34;resourceIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">52</span> <span class="s2">&#34;c084c3cb-2506-4477-b738-6debd7ac3827&#34;</span><span class="p">,</span>
<span class="ln">53</span> <span class="s2">&#34;483de42b-6a13-437e-90f9-e1e7a9cbc158&#34;</span>
<span class="ln">54</span> <span class="p">]</span>
<span class="ln">55</span><span class="p">}</span>
</code></pre></div><p>The second run output is simpler, as the public VMs have only a single interface:</p>
<div class="highlight"><pre class="chroma"><code class="language-json" data-lang="json"><span class="ln"> 1</span><span class="p">{</span>
<span class="ln"> 2</span> <span class="nt">&#34;componentId&#34;</span><span class="p">:</span> <span class="s2">&#34;public&#34;</span><span class="p">,</span>
<span class="ln"> 3</span> <span class="nt">&#34;endpointId&#34;</span><span class="p">:</span> <span class="s2">&#34;d098094c-5e88-4428-bcca-83e5ab6e9f6b&#34;</span><span class="p">,</span>
<span class="ln"> 4</span> <span class="nt">&#34;externalIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln"> 5</span> <span class="s2">&#34;public-344&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">first</span> <span class="err">VM</span> <span class="err">hostname</span>
<span class="ln"> 6</span> <span class="s2">&#34;public-345&#34;</span> <span class="err">//</span> <span class="err">second</span> <span class="err">VM</span> <span class="err">hostname</span>
<span class="ln"> 7</span> <span class="p">],</span>
<span class="ln"> 8</span> <span class="nt">&#34;blueprintId&#34;</span><span class="p">:</span> <span class="s2">&#34;d2ce10f3-6671-4588-9bf7-9d4b47085865&#34;</span><span class="p">,</span>
<span class="ln"> 9</span> <span class="nt">&#34;tags&#34;</span><span class="p">:</span> <span class="p">{</span>
<span class="ln">10</span> <span class="nt">&#34;project&#34;</span><span class="p">:</span> <span class="s2">&#34;Prod1&#34;</span>
<span class="ln">11</span> <span class="p">},</span>
<span class="ln">12</span> <span class="nt">&#34;customProperties&#34;</span><span class="p">:</span> <span class="p">{</span>
<span class="ln">13</span> <span class="nt">&#34;flavor&#34;</span><span class="p">:</span> <span class="s2">&#34;small&#34;</span><span class="p">,</span>
<span class="ln">14</span> <span class="nt">&#34;neglectPowerOffVms&#34;</span><span class="p">:</span> <span class="s2">&#34;false&#34;</span><span class="p">,</span>
<span class="ln">15</span> <span class="nt">&#34;image&#34;</span><span class="p">:</span> <span class="s2">&#34;photon&#34;</span><span class="p">,</span>
<span class="ln">16</span> <span class="nt">&#34;zone_overlapping_migrated&#34;</span><span class="p">:</span> <span class="s2">&#34;true&#34;</span><span class="p">,</span>
<span class="ln">17</span> <span class="nt">&#34;count&#34;</span><span class="p">:</span> <span class="s2">&#34;2&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">two</span> <span class="err">VMs</span> <span class="err">requested</span>
<span class="ln">18</span> <span class="nt">&#34;project&#34;</span><span class="p">:</span> <span class="s2">&#34;ffab9ffd-a072-4617-bbb4-54101b3b5dd1&#34;</span><span class="p">,</span>
<span class="ln">19</span> <span class="nt">&#34;flavorMappingName&#34;</span><span class="p">:</span> <span class="s2">&#34;small&#34;</span><span class="p">,</span>
<span class="ln">20</span> <span class="nt">&#34;isSimulate&#34;</span><span class="p">:</span> <span class="s2">&#34;false&#34;</span>
<span class="ln">21</span> <span class="p">},</span>
<span class="ln">22</span> <span class="nt">&#34;networkProfileIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">23</span> <span class="s2">&#34;14727fec-33cf-437d-8a20-91dcffef2717&#34;</span><span class="p">,</span> <span class="err">//</span> <span class="err">first</span> <span class="err">VM</span> <span class="err">selected</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">24</span> <span class="s2">&#34;14727fec-33cf-437d-8a20-91dcffef2717&#34;</span> <span class="err">//</span> <span class="err">second</span> <span class="err">VM</span> <span class="err">selected</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">25</span> <span class="p">],</span>
<span class="ln">26</span> <span class="nt">&#34;componentTypeId&#34;</span><span class="p">:</span> <span class="s2">&#34;Cloud.vSphere.Machine&#34;</span><span class="p">,</span>
<span class="ln">27</span> <span class="nt">&#34;requestId&#34;</span><span class="p">:</span> <span class="s2">&#34;3bdde4cf-3b12-4b43-90d8-7ccf2c47b1b6&#34;</span><span class="p">,</span>
<span class="ln">28</span> <span class="nt">&#34;deploymentId&#34;</span><span class="p">:</span> <span class="s2">&#34;fa4b9df4-9664-474a-a3f2-056db527e6e3&#34;</span><span class="p">,</span>
<span class="ln">29</span> <span class="nt">&#34;zoneId&#34;</span><span class="p">:</span> <span class="s2">&#34;d8666ddd-804e-4587-a37e-58b72708743d&#34;</span><span class="p">,</span>
<span class="ln">30</span> <span class="nt">&#34;networkSelectionIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">31</span> <span class="p">[</span> <span class="err">//</span> <span class="err">first</span> <span class="err">VM</span>
<span class="ln">32</span> <span class="p">[</span> <span class="err">//</span> <span class="err">first</span> <span class="err">NIC</span>
<span class="ln">33</span> <span class="s2">&#34;a9ec7502-5fb6-45fc-9f99-13f946a28b8f&#34;</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">34</span> <span class="p">]</span>
<span class="ln">35</span> <span class="p">],</span>
<span class="ln">36</span> <span class="p">[</span> <span class="err">//</span> <span class="err">second</span> <span class="err">VM</span>
<span class="ln">37</span> <span class="p">[</span> <span class="err">//</span> <span class="err">first</span> <span class="err">NIC</span>
<span class="ln">38</span> <span class="s2">&#34;a9ec7502-5fb6-45fc-9f99-13f946a28b8f&#34;</span> <span class="err">//</span> <span class="err">matching</span> <span class="err">network</span> <span class="err">in</span> <span class="err">network</span> <span class="err">profile</span>
<span class="ln">39</span> <span class="p">]</span>
<span class="ln">40</span> <span class="p">]</span>
<span class="ln">41</span> <span class="p">],</span>
<span class="ln">42</span> <span class="nt">&#34;projectId&#34;</span><span class="p">:</span> <span class="s2">&#34;ffab9ffd-a072-4617-bbb4-54101b3b5dd1&#34;</span><span class="p">,</span>
<span class="ln">43</span> <span class="nt">&#34;resourceIds&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">44</span> <span class="s2">&#34;e3a981a0-80fa-4b39-9c06-7c4d021b5f08&#34;</span><span class="p">,</span>
<span class="ln">45</span> <span class="s2">&#34;c1a79ccf-4600-4bb2-9ff9-5babdd63df40&#34;</span>
<span class="ln">46</span> <span class="p">]</span>
<span class="ln">47</span><span class="p">}</span>
</code></pre></div><h2 id="matching-networks-against-ipam">Matching networks against IPAM</h2>
<p>We need to get further information about the matching networks and find them in IPAM to make the allocation or deallocation. Using the IaaS API we can get the needed data.</p>
<p>For Aria API we use the <a href="https://marketplace.cloud.vmware.com/services/details/vmware-vrealize-orchestrator-plug-in-for-vrealize-automation011?slug=true">Orchestrator Plug-in for vRealize Automation</a>.</p>
<p>The EBS system user does not have Cloud Assembly access and won't be able to read IaaS API, so we create a service user for that purpose with Read permissions:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-datastore-custom-placement/apiuser.png" alt=""></p>
<p>and create a vRA endpoint within vRO:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-datastore-custom-placement/vra_endpoint.png" alt=""></p>
<p>To collect the network information of the sample subnet <tt>6bf0d8fb-6f39-4a43-b5d9-56a371030171</tt> we can run the following code:</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln">1</span><span class="kd">var</span> <span class="nx">vraHost</span> <span class="o">=</span> <span class="nx">VraHostManager</span><span class="p">.</span><span class="nx">findHostsByType</span><span class="p">(</span><span class="s2">&#34;vra-onprem&#34;</span><span class="p">).</span><span class="nx">filter</span><span class="p">(</span>
<span class="ln">2</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">host</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">3</span> <span class="k">return</span> <span class="nx">host</span><span class="p">.</span><span class="nx">name</span> <span class="o">==</span> <span class="s1">&#39;apiuser&#39;</span> <span class="c1">// find the vRA endpoint to use
</span><span class="ln">4</span><span class="c1"></span> <span class="p">}</span>
<span class="ln">5</span><span class="p">)[</span><span class="mi">0</span><span class="p">];</span>
<span class="ln">6</span>
<span class="ln">7</span><span class="kd">var</span> <span class="nx">fabricNetwork</span> <span class="o">=</span> <span class="nx">VraEntitiesFinder</span><span class="p">.</span><span class="nx">getFabricNetworks</span><span class="p">(</span><span class="nx">vraHost</span><span class="p">,</span> <span class="s2">&#34;id eq 6bf0d8fb-6f39-4a43-b5d9-56a371030171&#34;</span><span class="p">)[</span><span class="mi">0</span><span class="p">];</span>
<span class="ln">8</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">fabricNetwork</span><span class="p">);</span>
</code></pre></div><p>The output:
<div class="highlight"><pre class="chroma"><code class="language-text" data-lang="text"><span class="ln"> 1</span>DynamicWrapper (Instance) : [VraFabricNetwork]-[class com.vmware.o11n.plugin.vra_gen.FabricNetwork_Wrapper] -- VALUE : class FabricNetwork {
<span class="ln"> 2</span> owner: null
<span class="ln"> 3</span> links: {cloud-accounts=class Href {
<span class="ln"> 4</span> hrefs: [/iaas/api/cloud-accounts/d098094c-5e88-4428-bcca-83e5ab6e9f6b]
<span class="ln"> 5</span> href: null
<span class="ln"> 6</span> }, network-domain=class Href {
<span class="ln"> 7</span> hrefs: null
<span class="ln"> 8</span> href: /iaas/api/network-domains/c9932000429de93c72785683eb33e47d74936aaa
<span class="ln"> 9</span> }, self=class Href {
<span class="ln">10</span> hrefs: null
<span class="ln">11</span> href: /iaas/api/fabric-networks/6bf0d8fb-6f39-4a43-b5d9-56a371030171
<span class="ln">12</span> }, region=class Href {
<span class="ln">13</span> hrefs: null
<span class="ln">14</span> href: /iaas/api/regions/850e477a-8d22-43bb-9a2e-f80aa1daafb6
<span class="ln">15</span> }}
<span class="ln">16</span> externalRegionId: Datacenter:datacenter-2
<span class="ln">17</span> description: null
<span class="ln">18</span> externalId: DistributedVirtualPortgroup:dvportgroup-49872
<span class="ln">19</span> orgId: 360bc331-1360-4d76-8000-aae9f7491989
<span class="hl"><span class="ln">20</span> tags: [class Tag {
</span><span class="hl"><span class="ln">21</span> value: lan
</span><span class="hl"><span class="ln">22</span> key: zone
</span><span class="hl"><span class="ln">23</span> }]
</span><span class="ln">24</span> createdAt: 2023-08-15
<span class="ln">25</span> ipv6Cidr: null
<span class="ln">26</span> cloudAccountIds: [d098094c-5e88-4428-bcca-83e5ab6e9f6b]
<span class="ln">27</span> isDefault: null
<span class="ln">28</span> customProperties: {}
<span class="hl"><span class="ln">29</span> name: net1-pr-nsxt-ls
</span><span class="ln">30</span> isPublic: null
<span class="hl"><span class="ln">31</span> cidr: 10.32.101.0/24
</span><span class="ln">32</span> id: 6bf0d8fb-6f39-4a43-b5d9-56a371030171
<span class="ln">33</span> updatedAt: 2023-12-11
<span class="ln">34</span>}</code></pre></div></p>
<p>As hightlighted, we can search in IPAM by network name, network CIDR or even by tags if we want.</p>
<h2 id="phpipam-functions">phpIPAM Functions</h2>
<p>The <a href="https://phpipam.net/api/api_documentation/">phpIPAM documentation</a> explains how to use this IPAM solution via API. We'll need an app_id and a user in phpIPAM. To receive and send data via the API a token must be received and included in further REST client calls. We implement this with 2 functions:</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln"> 1</span><span class="kd">function</span> <span class="nx">getIPAMtoken</span><span class="p">(</span><span class="nx">restHost</span><span class="p">)</span> <span class="p">{</span>
<span class="ln"> 2</span> <span class="kd">var</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">restHost</span><span class="p">.</span><span class="nx">createRequest</span><span class="p">(</span><span class="s2">&#34;POST&#34;</span><span class="p">,</span> <span class="s2">&#34;/user/&#34;</span><span class="p">,</span> <span class="s2">&#34;&#34;</span><span class="p">);</span>
<span class="ln"> 3</span> <span class="nx">request</span><span class="p">.</span><span class="nx">contentType</span> <span class="o">=</span> <span class="s2">&#34;application/json&#34;</span><span class="p">;</span>
<span class="ln"> 4</span> <span class="kd">var</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">request</span><span class="p">.</span><span class="nx">execute</span><span class="p">();</span>
<span class="ln"> 5</span> <span class="k">if</span> <span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">statusCode</span> <span class="o">&lt;</span> <span class="mi">400</span><span class="p">)</span> <span class="k">return</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">contentAsString</span><span class="p">).</span><span class="nx">data</span><span class="p">.</span><span class="nx">token</span><span class="p">;</span>
<span class="ln"> 6</span> <span class="k">else</span> <span class="k">throw</span> <span class="s2">&#34;phpIPAM token error:&#34;</span> <span class="o">+</span> <span class="nx">response</span><span class="p">.</span><span class="nx">contentAsString</span><span class="p">;</span>
<span class="ln"> 7</span><span class="p">}</span>
<span class="ln"> 8</span>
<span class="ln"> 9</span><span class="kd">function</span> <span class="nx">IPAMrequest</span><span class="p">(</span><span class="nx">restHost</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">path</span><span class="p">,</span> <span class="nx">method</span><span class="p">,</span> <span class="nx">data</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">10</span> <span class="kd">var</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">restHost</span><span class="p">.</span><span class="nx">createRequest</span><span class="p">(</span><span class="nx">method</span><span class="p">,</span> <span class="nx">path</span><span class="p">,</span> <span class="nx">data</span> <span class="o">?</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="o">:</span> <span class="s2">&#34;&#34;</span><span class="p">);</span>
<span class="ln">11</span> <span class="nx">request</span><span class="p">.</span><span class="nx">contentType</span> <span class="o">=</span> <span class="s2">&#34;application/json&#34;</span><span class="p">;</span>
<span class="ln">12</span> <span class="nx">request</span><span class="p">.</span><span class="nx">setHeader</span><span class="p">(</span><span class="s2">&#34;phpipam-token&#34;</span><span class="p">,</span> <span class="nx">token</span><span class="p">);</span>
<span class="ln">13</span> <span class="kd">var</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">request</span><span class="p">.</span><span class="nx">execute</span><span class="p">();</span>
<span class="ln">14</span> <span class="nx">System</span><span class="p">.</span><span class="nx">debug</span><span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">contentAsString</span><span class="p">);</span>
<span class="ln">15</span> <span class="k">try</span> <span class="p">{</span>
<span class="ln">16</span> <span class="k">return</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">contentAsString</span><span class="p">);</span>
<span class="ln">17</span> <span class="p">}</span>
<span class="ln">18</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">19</span> <span class="k">throw</span> <span class="s2">&#34;phpIPAM request error:&#34;</span> <span class="o">+</span> <span class="nx">response</span><span class="p">.</span><span class="nx">contentAsString</span><span class="p">;</span>
<span class="ln">20</span> <span class="p">}</span>
<span class="ln">21</span><span class="p">}</span>
</code></pre></div><p>The REST host is registered with Basic Authentication:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/resthost.png" alt=""><br></p>
<p>We can search for a subnet by CIDR on <tt>/api/app_id/subnets/cidr/{subnetCIDR}</tt>, and allocate an IP on <tt>/api/app_id/addresses/first_free/{subnetId}</tt>.</p>
<h2 id="expected-workflow-output">Expected workflow output</h2>
<p>The IP allocation workflow can return quite a few parameters, we chose to implement 2 of them: setting the IP addresses by <tt>addresses</tt></p>
<div class="highlight"><pre class="chroma"><code class="language-json" data-lang="json"><span class="ln"> 1</span><span class="p">[</span>
<span class="ln"> 2</span> <span class="p">[</span>
<span class="ln"> 3</span> <span class="s2">&#34;10.32.102.34&#34;</span><span class="p">,</span>
<span class="ln"> 4</span> <span class="s2">&#34;10.32.103.31&#34;</span>
<span class="ln"> 5</span> <span class="p">],</span>
<span class="ln"> 6</span> <span class="p">[</span>
<span class="ln"> 7</span> <span class="s2">&#34;10.32.102.35&#34;</span><span class="p">,</span>
<span class="ln"> 8</span> <span class="s2">&#34;10.32.103.32&#34;</span>
<span class="ln"> 9</span> <span class="p">]</span>
<span class="ln">10</span><span class="p">]</span>
</code></pre></div><p>and selecting the target network by <tt>networkSelectionIds</tt>.</p>
<div class="highlight"><pre class="chroma"><code class="language-json" data-lang="json"><span class="ln"> 1</span><span class="p">[</span>
<span class="ln"> 2</span> <span class="p">[</span>
<span class="ln"> 3</span> <span class="p">[</span>
<span class="ln"> 4</span> <span class="s2">&#34;c2168e8f-62aa-45e2-9dd9-502d22359214&#34;</span>
<span class="ln"> 5</span> <span class="p">],</span>
<span class="ln"> 6</span> <span class="p">[</span>
<span class="ln"> 7</span> <span class="s2">&#34;dd65009b-3900-46ee-a6a2-d3226d755f4b&#34;</span>
<span class="ln"> 8</span> <span class="p">]</span>
<span class="ln"> 9</span> <span class="p">],</span>
<span class="ln">10</span> <span class="p">[</span>
<span class="ln">11</span> <span class="p">[</span>
<span class="ln">12</span> <span class="s2">&#34;c2168e8f-62aa-45e2-9dd9-502d22359214&#34;</span>
<span class="ln">13</span> <span class="p">],</span>
<span class="ln">14</span> <span class="p">[</span>
<span class="ln">15</span> <span class="s2">&#34;dd65009b-3900-46ee-a6a2-d3226d755f4b&#34;</span>
<span class="ln">16</span> <span class="p">]</span>
<span class="ln">17</span> <span class="p">]</span>
<span class="ln">18</span><span class="p">]</span>
</code></pre></div><h2 id="putting-ip-allocate-code-all-together">Putting IP Allocate code all together</h2>
<p>Let's see the IP allocate workflow code. We have 3 nested loops: the outer is looping through the VMs, the next one is looping through the NICs and the inner one is looping through the networks and trying to get an IP. If the IP is granted, we quit the inner loop, otherwise we try the other networks selected in the network profile.</p>
<p>We assume that only a single IPAM network is found for a specific CIDR.</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln"> 1</span><span class="kd">var</span> <span class="nx">vraHost</span> <span class="o">=</span> <span class="nx">VraHostManager</span><span class="p">.</span><span class="nx">findHostsByType</span><span class="p">(</span><span class="s2">&#34;vra-onprem&#34;</span><span class="p">).</span><span class="nx">filter</span><span class="p">(</span>
<span class="ln"> 2</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">host</span><span class="p">)</span> <span class="p">{</span>
<span class="ln"> 3</span> <span class="k">return</span> <span class="nx">host</span><span class="p">.</span><span class="nx">name</span> <span class="o">==</span> <span class="s1">&#39;apiuser&#39;</span> <span class="c1">// find the vRA endpoint to use
</span><span class="ln"> 4</span><span class="c1"></span> <span class="p">}</span>
<span class="ln"> 5</span><span class="p">)[</span><span class="mi">0</span><span class="p">];</span>
<span class="ln"> 6</span>
<span class="ln"> 7</span><span class="kd">var</span> <span class="nx">ipamHost</span> <span class="o">=</span> <span class="nx">RESTHostManager</span><span class="p">.</span><span class="nx">getHosts</span><span class="p">()</span>
<span class="ln"> 8</span> <span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">hostid</span><span class="p">)</span> <span class="p">{</span><span class="k">return</span> <span class="nx">RESTHostManager</span><span class="p">.</span><span class="nx">getHost</span><span class="p">(</span><span class="nx">hostid</span><span class="p">)})</span>
<span class="ln"> 9</span> <span class="p">.</span><span class="nx">filter</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">host</span><span class="p">)</span> <span class="p">{</span><span class="k">return</span> <span class="s2">&#34;phpIPAM&#34;</span> <span class="o">==</span> <span class="nx">host</span><span class="p">.</span><span class="nx">name</span><span class="p">})[</span><span class="mi">0</span><span class="p">];</span>
<span class="ln">10</span>
<span class="ln">11</span><span class="kd">var</span> <span class="nx">ipamToken</span> <span class="o">=</span> <span class="nx">getIPAMtoken</span><span class="p">(</span><span class="nx">ipamHost</span><span class="p">);</span>
<span class="ln">12</span>
<span class="ln">13</span><span class="nx">addresses</span> <span class="o">=</span> <span class="p">[];</span>
<span class="ln">14</span><span class="nx">networkSelectionIds</span> <span class="o">=</span> <span class="p">[];</span>
<span class="ln">15</span>
<span class="ln">16</span><span class="k">for</span> <span class="p">(</span><span class="nx">vmIndex</span> <span class="k">in</span> <span class="nx">inputProperties</span><span class="p">.</span><span class="nx">resourceIds</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">17</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Processing VM &#34;</span> <span class="o">+</span> <span class="nx">inputProperties</span><span class="p">.</span><span class="nx">externalIds</span><span class="p">[</span><span class="nx">vmIndex</span><span class="p">]);</span>
<span class="ln">18</span> <span class="nx">addresses</span><span class="p">.</span><span class="nx">push</span><span class="p">([]);</span>
<span class="ln">19</span> <span class="nx">networkSelectionIds</span><span class="p">.</span><span class="nx">push</span><span class="p">([]);</span>
<span class="ln">20</span>
<span class="ln">21</span> <span class="k">for</span> <span class="p">(</span><span class="nx">nicIndex</span> <span class="k">in</span> <span class="nx">inputProperties</span><span class="p">.</span><span class="nx">networkSelectionIds</span><span class="p">[</span><span class="nx">vmIndex</span><span class="p">])</span> <span class="p">{</span>
<span class="ln">22</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;NIC&#34;</span> <span class="o">+</span> <span class="nx">nicIndex</span><span class="p">);</span>
<span class="ln">23</span> <span class="k">for</span> <span class="nx">each</span> <span class="p">(</span><span class="nx">networkId</span> <span class="k">in</span> <span class="nx">inputProperties</span><span class="p">.</span><span class="nx">networkSelectionIds</span><span class="p">[</span><span class="nx">vmIndex</span><span class="p">][</span><span class="nx">nicIndex</span><span class="p">])</span> <span class="p">{</span>
<span class="ln">24</span> <span class="kd">var</span> <span class="nx">fabricNetwork</span> <span class="o">=</span> <span class="nx">VraEntitiesFinder</span><span class="p">.</span><span class="nx">getFabricNetworks</span><span class="p">(</span><span class="nx">vraHost</span><span class="p">,</span> <span class="s2">&#34;id eq &#34;</span> <span class="o">+</span> <span class="nx">networkId</span><span class="p">)[</span><span class="mi">0</span><span class="p">];</span>
<span class="ln">25</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Checking network &#34;</span> <span class="o">+</span> <span class="nx">fabricNetwork</span><span class="p">.</span><span class="nx">name</span><span class="p">);</span>
<span class="ln">26</span> <span class="kd">var</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">IPAMrequest</span><span class="p">(</span><span class="nx">ipamHost</span><span class="p">,</span> <span class="nx">ipamToken</span><span class="p">,</span> <span class="s2">&#34;/subnets/cidr/&#34;</span> <span class="o">+</span> <span class="nx">fabricNetwork</span><span class="p">.</span><span class="nx">cidr</span><span class="p">,</span> <span class="s2">&#34;GET&#34;</span><span class="p">);</span>
<span class="ln">27</span> <span class="k">if</span> <span class="p">(</span><span class="mi">200</span> <span class="o">==</span> <span class="nx">response</span><span class="p">.</span><span class="nx">code</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">28</span> <span class="kd">var</span> <span class="nx">subnetId</span> <span class="o">=</span> <span class="nx">response</span><span class="p">.</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span><span class="p">;</span>
<span class="ln">29</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Subnet &#34;</span> <span class="o">+</span> <span class="nx">fabricNetwork</span><span class="p">.</span><span class="nx">cidr</span> <span class="o">+</span> <span class="s2">&#34; found in IPAM with ID &#34;</span> <span class="o">+</span> <span class="nx">subnetId</span><span class="p">);</span>
<span class="ln">30</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">IPAMrequest</span><span class="p">(</span><span class="nx">ipamHost</span><span class="p">,</span> <span class="nx">ipamToken</span><span class="p">,</span> <span class="s2">&#34;/addresses/first_free/&#34;</span> <span class="o">+</span> <span class="nx">subnetId</span><span class="p">,</span> <span class="s2">&#34;POST&#34;</span><span class="p">,</span>
<span class="ln">31</span> <span class="p">{</span><span class="nx">hostname</span><span class="o">:</span> <span class="nx">inputProperties</span><span class="p">.</span><span class="nx">externalIds</span><span class="p">[</span><span class="nx">vmIndex</span><span class="p">]});</span>
<span class="ln">32</span> <span class="k">if</span> <span class="p">(</span><span class="mi">201</span> <span class="o">==</span> <span class="nx">response</span><span class="p">.</span><span class="nx">code</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">33</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Reserved IP &#34;</span> <span class="o">+</span> <span class="nx">response</span><span class="p">.</span><span class="nx">data</span><span class="p">);</span>
<span class="ln">34</span> <span class="nx">addresses</span><span class="p">[</span><span class="nx">vmIndex</span><span class="p">].</span><span class="nx">push</span><span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">data</span><span class="p">);</span>
<span class="ln">35</span> <span class="nx">networkSelectionIds</span><span class="p">[</span><span class="nx">vmIndex</span><span class="p">].</span><span class="nx">push</span><span class="p">([</span><span class="nx">networkId</span><span class="p">]);</span>
<span class="ln">36</span> <span class="k">break</span><span class="p">;</span>
<span class="ln">37</span> <span class="p">}</span>
<span class="ln">38</span> <span class="k">else</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;No free IP found&#34;</span><span class="p">)</span>
<span class="ln">39</span> <span class="p">}</span>
<span class="ln">40</span> <span class="k">else</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Subnet &#34;</span> <span class="o">+</span> <span class="nx">fabricNetwork</span><span class="p">.</span><span class="nx">cidr</span> <span class="o">+</span> <span class="s2">&#34; not found in IPAM&#34;</span><span class="p">);</span>
<span class="ln">41</span> <span class="p">}</span>
<span class="ln">42</span> <span class="p">}</span>
<span class="ln">43</span>
<span class="ln">44</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;---&#34;</span><span class="p">)</span>
<span class="ln">45</span><span class="p">}</span>
<span class="ln">46</span>
<span class="ln">47</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">networkSelectionIds</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">));</span>
<span class="ln">48</span><span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">addresses</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">));</span>
</code></pre></div><h2 id="ip-release">IP Release</h2>
<p>Releasing is much simpler. The EBS workflow is run by VM and we need only the IPs from the inputProperties:</p>
<div class="highlight"><pre class="chroma"><code class="language-json" data-lang="json"><span class="ln">1</span><span class="p">{</span>
<span class="ln">2</span> <span class="nt">&#34;addresses&#34;</span><span class="p">:</span> <span class="p">[</span>
<span class="ln">3</span> <span class="p">[</span>
<span class="ln">4</span> <span class="s2">&#34;10.32.102.34&#34;</span><span class="p">,</span>
<span class="ln">5</span> <span class="s2">&#34;10.32.103.31&#34;</span>
<span class="ln">6</span> <span class="p">]</span>
<span class="ln">7</span> <span class="p">],</span>
<span class="ln">8</span> <span class="nt">&#34;componentId&#34;</span><span class="p">:</span> <span class="s2">&#34;internal[0]&#34;</span><span class="p">,</span>
<span class="ln">9</span><span class="err">...</span>
</code></pre></div><p>The code (no workflow outputs):</p>
<div class="highlight"><pre class="chroma"><code class="language-javascript" data-lang="javascript"><span class="ln"> 1</span><span class="kd">var</span> <span class="nx">ipamHost</span> <span class="o">=</span> <span class="nx">RESTHostManager</span><span class="p">.</span><span class="nx">getHosts</span><span class="p">()</span>
<span class="ln"> 2</span> <span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">hostid</span><span class="p">)</span> <span class="p">{</span><span class="k">return</span> <span class="nx">RESTHostManager</span><span class="p">.</span><span class="nx">getHost</span><span class="p">(</span><span class="nx">hostid</span><span class="p">)})</span>
<span class="ln"> 3</span> <span class="p">.</span><span class="nx">filter</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">host</span><span class="p">)</span> <span class="p">{</span><span class="k">return</span> <span class="s2">&#34;phpIPAM&#34;</span> <span class="o">==</span> <span class="nx">host</span><span class="p">.</span><span class="nx">name</span><span class="p">})[</span><span class="mi">0</span><span class="p">];</span>
<span class="ln"> 4</span>
<span class="ln"> 5</span><span class="kd">var</span> <span class="nx">ipamToken</span> <span class="o">=</span> <span class="nx">getIPAMtoken</span><span class="p">(</span><span class="nx">ipamHost</span><span class="p">);</span>
<span class="ln"> 6</span>
<span class="ln"> 7</span><span class="k">for</span> <span class="nx">each</span> <span class="p">(</span><span class="nx">ip</span> <span class="k">in</span> <span class="nx">inputProperties</span><span class="p">.</span><span class="nx">addresses</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="p">{</span>
<span class="ln"> 8</span> <span class="kd">var</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">IPAMrequest</span><span class="p">(</span><span class="nx">ipamHost</span><span class="p">,</span> <span class="nx">ipamToken</span><span class="p">,</span> <span class="s2">&#34;/addresses/search/&#34;</span> <span class="o">+</span> <span class="nx">ip</span><span class="p">,</span> <span class="s2">&#34;GET&#34;</span><span class="p">);</span>
<span class="ln"> 9</span> <span class="k">if</span> <span class="p">(</span><span class="mi">200</span> <span class="o">==</span> <span class="nx">response</span><span class="p">.</span><span class="nx">code</span><span class="p">)</span> <span class="p">{</span>
<span class="ln">10</span> <span class="kd">var</span> <span class="nx">addressId</span> <span class="o">=</span> <span class="nx">response</span><span class="p">.</span><span class="nx">data</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span><span class="p">;</span>
<span class="ln">11</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;IP &#34;</span> <span class="o">+</span> <span class="nx">ip</span> <span class="o">+</span> <span class="s2">&#34; found in IPAM with ID &#34;</span> <span class="o">+</span> <span class="nx">addressId</span><span class="p">);</span>
<span class="ln">12</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">IPAMrequest</span><span class="p">(</span><span class="nx">ipamHost</span><span class="p">,</span> <span class="nx">ipamToken</span><span class="p">,</span> <span class="s2">&#34;/addresses/&#34;</span> <span class="o">+</span> <span class="nx">addressId</span><span class="p">,</span> <span class="s2">&#34;DELETE&#34;</span><span class="p">);</span>
<span class="ln">13</span> <span class="k">if</span> <span class="p">(</span><span class="mi">200</span> <span class="o">==</span> <span class="nx">response</span><span class="p">.</span><span class="nx">code</span><span class="p">)</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Released IP &#34;</span> <span class="o">+</span> <span class="nx">ip</span><span class="p">);</span>
<span class="ln">14</span> <span class="k">else</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;Could not release IP &#34;</span> <span class="o">+</span> <span class="nx">ip</span> <span class="o">+</span> <span class="s2">&#34; (&#34;</span> <span class="o">+</span> <span class="nx">response</span><span class="p">.</span><span class="nx">contentAsString</span> <span class="o">+</span> <span class="s2">&#34;)&#34;</span><span class="p">);</span>
<span class="ln">15</span> <span class="p">}</span>
<span class="ln">16</span> <span class="k">else</span> <span class="nx">System</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">&#34;IP &#34;</span> <span class="o">+</span> <span class="nx">ip</span> <span class="o">+</span> <span class="s2">&#34; not found in IPAM&#34;</span><span class="p">);</span>
<span class="ln">17</span><span class="p">}</span>
</code></pre></div><h2 id="conclusion">Conclusion</h2>
<p>In less than 100 lines of code we could implement a fully functional phpIPAM integration:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vra8-ipam-in-vro/addresses.png" alt=""><br></p>
<p>Bear in mind this is just a sample code, not production ready (for the sake of simplicity). Error handling needs to be enhanced.
Download the <em>com.test.phpipam</em> package containing the workflows from GitHub: <a href="https://github.com/kuklis/vro8-packages">https://github.com/kuklis/vro8-packages</a></p>
</description>
</item>
<item>
<title>Get pyodbc working within Aria Automation Orchestrator</title>
<link>https://kuklis.github.io/cma/post/vro8-pyodbc/</link>
<pubDate>Fri, 20 Oct 2023 00:00:00 +0000</pubDate>
<guid>https://kuklis.github.io/cma/post/vro8-pyodbc/</guid>
<description>
<p>Using the Python module pyodbc with unixODBC to connect to an MS SQL Server database</p>
<div class="toc">
<H3>Table of Contents</H3>
<nav id="TableOfContents">
<ul>
<li><a href="#preface">Preface</a></li>
<li><a href="#photon-os-4">Photon OS 4</a></li>
<li><a href="#install-dependencies-on-the-local-vm">Install dependencies on the local VM</a></li>
<li><a href="#test-pyodbc-functionality-on-the-local-vm">Test pyodbc functionality on the local VM</a></li>
<li><a href="#packaging-into-an-orchestrator-action">Packaging into an Orchestrator action</a></li>
<li><a href="#trying-the-action">Trying the action</a></li>
<li><a href="#final-thoughts">Final thoughts</a></li>
</ul>
</nav>
</div>
<h2 id="preface">Preface</h2>
<p>pyodbc is an open source Python module that makes accessing ODBC databases simple. Aria Automation Orchestrator provides Python runtime environment that can be extended with additional modules. However, pyodbc requires some packages on the host OS to function properly, and we found these components are missing from the underlying container image so pyodbc does not work out of box.</p>
<p>Our goal is to fill the gaps and make this module working.</p>
<h2 id="photon-os-4">Photon OS 4</h2>
<p>During my initial attempts of to run pyodbc code I received some weird error messages from the Python runtime. I followed the official documentation on how to <a href="https://docs.vmware.com/en/vRealize-Automation/8.11/Using-and-Managing-Cloud-Assembly/GUID-CC6DEEF1-49E8-4881-82A6-FA10DC0135E8.html">Create a ZIP package for Python runtime extensibility actions</a>, but still received this error:</p>
<blockquote>
<p><tt>ModuleNotFoundError: No module named 'pyodbc'</tt></p>
</blockquote>
<p>After searching on Stackoverflow I realized my problem might be having a different Python runtime version locally. I also found out that I'll have to install some additional OS packages, so decided to install the same OS and continue the preparations on this virtual machine.</p>
<p>The container images providing Python runtime based on Photon OS. To get the exact version and packages installed, let's run the following code:</p>
<div class="highlight"><pre class="chroma"><code class="language-python" data-lang="python"><span class="ln">1</span><span class="kn">import</span> <span class="nn">os</span>
<span class="ln">2</span><span class="k">def</span> <span class="nf">handler</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">inputs</span><span class="p">):</span>
<span class="ln">3</span> <span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="s2">&#34;cat /etc/photon-release&#34;</span><span class="p">)</span>
<span class="ln">4</span> <span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="s2">&#34;tdnf --disablerepo=* info|grep &#39;Name :&#39;&#34;</span><span class="p">)</span>
<span class="ln">5</span> <span class="k">print</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="p">)</span>
<span class="ln">6</span> <span class="k">return</span> <span class="s2">&#34;done&#34;</span>
</code></pre></div><p><img src="https://kuklis.github.io/cma/cma/img/vro8-pyodbc/rpmlist.png" alt=""></p>
<p>Let's install this build <a href="https://github.com/vmware/photon/wiki/Downloading-Photon-OS#downloading-photon-os-40-rev2">Photon OS 4.0 Rev2</a> and login to the VM.</p>
<h2 id="install-dependencies-on-the-local-vm">Install dependencies on the local VM</h2>
<p>The first goal is to figure out what needs to be installed and configured to run pyodbc with Microsoft ODBC driver 18. Following the <a href="https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server">official documentation</a> we need to install unixODBC and msodbcsql18 packages. Unfortunately the latter is not released for Photon OS but we try which binary works for us.</p>
<p>Let's start with unixODBC and add zip package too.</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span>tdnf -y install unixODBC zip
</code></pre></div><p>Then install MS ODBC for SLES15 (this is what I found working).</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span>mkdir unixODBC
<span class="ln">2</span>curl -sSL https://packages.microsoft.com/sles/15/prod/Packages/m/msodbcsql18-18.3.2.1-1.x86_64.rpm -O --output-dir unixODBC
<span class="ln">3</span><span class="nv">ACCEPT_EULA</span><span class="o">=</span>Y tdnf install --nogpgcheck -y unixODBC/msodbcsql18-18.3.2.1-1.x86_64.rpm
</code></pre></div><p>To install pip we need to download the installer first, as it is not available in Photon repos.</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span>curl -sS https://files.pythonhosted.org/packages/e0/63/b428aaca15fcd98c39b07ca7149e24bc14205ad0f1c80ba2b01835aedde1/pip-23.3-py3-none-any.whl -O
<span class="ln">2</span>python pip-23.3-py3-none-any.whl/pip install --no-index pip-23.3-py3-none-any.whl
</code></pre></div><p>Now we can install pyodbc.</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span>pip install pyodbc
</code></pre></div><h2 id="test-pyodbc-functionality-on-the-local-vm">Test pyodbc functionality on the local VM</h2>
<p>Let's try to connect to our MS SQL server and run a query.</p>
<div class="highlight"><pre class="chroma"><code class="language-python" data-lang="python"><span class="ln"> 1</span><span class="kn">import</span> <span class="nn">pyodbc</span>
<span class="ln"> 2</span>
<span class="ln"> 3</span><span class="n">SERVER</span> <span class="o">=</span> <span class="s1">&#39;mssql.cloud.lab&#39;</span>
<span class="ln"> 4</span><span class="n">DATABASE</span> <span class="o">=</span> <span class="s1">&#39;test&#39;</span>
<span class="ln"> 5</span><span class="n">USERNAME</span> <span class="o">=</span> <span class="s1">&#39;sa&#39;</span>
<span class="ln"> 6</span><span class="n">PASSWORD</span> <span class="o">=</span> <span class="s1">&#39;P@ssw0rd&#39;</span>
<span class="ln"> 7</span>
<span class="ln"> 8</span><span class="n">connectionString</span> <span class="o">=</span> <span class="n">f</span><span class="s1">&#39;DRIVER={{ODBC Driver 18 for SQL Server}};SERVER={SERVER};DATABASE={DATABASE};UID={USERNAME};PWD={PASSWORD};TrustServerCertificate=yes;&#39;</span>
<span class="ln"> 9</span>
<span class="ln">10</span><span class="n">conn</span> <span class="o">=</span> <span class="n">pyodbc</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">connectionString</span><span class="p">)</span>
<span class="ln">11</span>
<span class="ln">12</span><span class="n">SQL_QUERY</span> <span class="o">=</span> <span class="s1">&#39;SELECT * FROM myObjects;&#39;</span>
<span class="ln">13</span>
<span class="ln">14</span><span class="n">cursor</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
<span class="ln">15</span><span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">SQL_QUERY</span><span class="p">)</span>
<span class="ln">16</span><span class="n">records</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchall</span><span class="p">()</span>
<span class="ln">17</span><span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">records</span><span class="p">:</span>
<span class="ln">18</span> <span class="k">print</span><span class="p">(</span><span class="n">r</span><span class="p">)</span>
</code></pre></div><p>The driver works, displaying the myObjects database table of our <a href="https://kuklis.github.io/cma/cma/post/vra8-custom-resources-1/">Custom Resource Type</a>:<br>
<img src="https://kuklis.github.io/cma/cma/img/vro8-pyodbc/testODBC.png" alt=""></p>
<h2 id="packaging-into-an-orchestrator-action">Packaging into an Orchestrator action</h2>
<p>We need to make sure all the dependencies are installed on the Python container as well. For that we'll copy the dependencies onto the runtime environment and use some OS commands introduced in <a href="https://kuklis.github.io/cma/cma/post/vro8-run-helm/">my previous post on Helm</a>.</p>
<p>Based on the RPM list the following packages are missing: <tt>unixODBC, libstdc++, glibc-iconv</tt><br>
Let's download them, with pyodbc, on our Photon OS VM:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span>tdnf reinstall -y --downloadonly --downloaddir unixODBC --disableexcludes unixODBC libstdc++ glibc-iconv
<span class="ln">2</span>pip install --no-cache-dir pyodbc --target<span class="o">=</span>unixODBC/lib
</code></pre></div><p>Let's extend our test Python script to install the packages prior to pyodbc usage. Only after the dependencies installation we can import pyodbc.</p>
<div class="highlight"><pre class="chroma"><code class="language-python" data-lang="python"><span class="ln"> 1</span><span class="kn">import</span> <span class="nn">os</span>
<span class="ln"> 2</span>
<span class="ln"> 3</span><span class="k">def</span> <span class="nf">handler</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">inputs</span><span class="p">):</span>
<span class="ln"> 4</span> <span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="s2">&#34;ACCEPT_EULA=Y tdnf --disablerepo=* --nogpgcheck -y install ./unixODBC-2.3.9-1.ph4.x86_64.rpm ./msodbcsql18-18.3.2.1-1.x86_64.rpm ./glibc-iconv-2.32-11.ph4.x86_64.rpm ./libstdc++-10.2.0-2.ph4.x86_64.rpm&#34;</span><span class="p">)</span>
<span class="ln"> 5</span>
<span class="ln"> 6</span> <span class="kn">import</span> <span class="nn">pyodbc</span>
<span class="ln"> 7</span> <span class="n">SERVER</span> <span class="o">=</span> <span class="s1">&#39;mssql.cloud.lab&#39;</span>
<span class="ln"> 8</span> <span class="n">DATABASE</span> <span class="o">=</span> <span class="s1">&#39;test&#39;</span>
<span class="ln"> 9</span> <span class="n">USERNAME</span> <span class="o">=</span> <span class="s1">&#39;sa&#39;</span>
<span class="ln">10</span> <span class="n">PASSWORD</span> <span class="o">=</span> <span class="s1">&#39;P@ssw0rd&#39;</span>
<span class="ln">11</span>
<span class="ln">12</span> <span class="n">connectionString</span> <span class="o">=</span> <span class="n">f</span><span class="s1">&#39;DRIVER={{ODBC Driver 18 for SQL Server}};SERVER={SERVER};DATABASE={DATABASE};UID={USERNAME};PWD={PASSWORD};TrustServerCertificate=yes;&#39;</span>
<span class="ln">13</span>
<span class="ln">14</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">pyodbc</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">connectionString</span><span class="p">)</span>
<span class="ln">15</span>
<span class="ln">16</span> <span class="n">SQL_QUERY</span> <span class="o">=</span> <span class="s1">&#39;SELECT * FROM dbo.myObjects;&#39;</span>
<span class="ln">17</span>
<span class="ln">18</span> <span class="n">cursor</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
<span class="ln">19</span> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">SQL_QUERY</span><span class="p">)</span>
<span class="ln">20</span> <span class="n">records</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchall</span><span class="p">()</span>
<span class="ln">21</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">records</span><span class="p">:</span>
<span class="ln">22</span> <span class="k">print</span><span class="p">(</span><span class="n">r</span><span class="p">)</span>
<span class="ln">23</span>
<span class="ln">24</span> <span class="k">return</span> <span class="s2">&#34;done&#34;</span>
</code></pre></div><p>After we saved this in <tt>unixODBC/handler.py</tt>, let's create the ZIP package to import the code into our Orchestrator action.</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span><span class="nb">cd</span> unixODBC
<span class="ln">2</span>zip -r ../unixODBC.zip .
</code></pre></div><h2 id="trying-the-action">Trying the action</h2>
<p>After creating the action let's try to run it:</p>
<p><img src="https://kuklis.github.io/cma/cma/img/vro8-pyodbc/pyodbc_logs.png" alt=""></p>
<p>As we can see in the logs, the packages are installed and then pyodbc can read from the MS SQL Server database.</p>
<h2 id="final-thoughts">Final thoughts</h2>
<p>Altough the SQL plugin of Orchestartor is capable of connecting SQL Servers, we cannot use it in Python actions. Python has a huge library of modules suitable for a lot of usecases we may need: the above method allows to use databases via the ODBC connector within Python code.</p>
<p>You can download the <em>com.test.unixodbc.mssql</em> package containing the code from GitHub: <a href="https://github.com/kuklis/vro8-packages">https://github.com/kuklis/vro8-packages</a></p>
</description>
</item>
<item>
<title>Custom day 2 actions for Tanzu in Aria Automation</title>
<link>https://kuklis.github.io/cma/post/vra8-tanzu-day2/</link>
<pubDate>Thu, 19 Oct 2023 00:00:00 +0000</pubDate>
<guid>https://kuklis.github.io/cma/post/vra8-tanzu-day2/</guid>
<description>
<p>Implementing a Kubernetes REST API client for Tanzu clusters</p>
<div class="toc">
<H3>Table of Contents</H3>
<nav id="TableOfContents">
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#authentication-by-client-certificates-within-orchestrator-plugin">Authentication by client certificates within Orchestrator plugin</a></li>
<li><a href="#downloading-the-certificates-with-python">Downloading the certificates with Python</a></li>
<li><a href="#kuberestclient-action">kubeRestClient action</a></li>
<li><a href="#sample-day-2-resource-action-create-namespace">Sample day 2 resource action: Create Namespace</a></li>
<li><a href="#testing-the-custom-action">Testing the custom action</a></li>
<li><a href="#summary">Summary</a></li>
</ul>
</nav>
</div>
<h2 id="introduction">Introduction</h2>
<p>In my previous post on <a href="https://kuklis.github.io/cma/cma/post/vro8-run-helm">Running Helm in vRO</a> we discussed that Aria Automation provides a certificate based authentication to the deployed Tanzu clusters via API at <a href="https://www.mgmt.cloud.vmware.com/cmx/webjars/swagger-ui/index.html#/Kubernetes%20Clusters/getKubeConfig_1">/cmx/api/resources/k8s/clusers/{id}/kube-config</a><br>
The same Kubeconfig can be downloaded on the GUI:<br>
<img src="https://kuklis.github.io/cma/cma/img/vra8-tanzu-day2/download_kubeconfig.png" alt=""></p>
<p>We will use the access provided by the certificates in this Kubeconfig to connect to the Kubernetes REST API of the cluster.</p>
<h2 id="authentication-by-client-certificates-within-orchestrator-plugin">Authentication by client certificates within Orchestrator plugin</h2>
<p>The HTTP REST plugin of vRO is capable of using client certificates as authentication method. However, this feature is <a href="https://docs.vmware.com/en/vRealize-Orchestrator/8.11/com.vmware.vrealize.orchestrator-use-plugins.doc/GUID-35F459BF-7EE6-469C-A3BB-AE0AF0E774AC.html#considerations-for-transient-hosts-1">not available for transient hosts</a>:</p>
<blockquote>
<ul>
<li>Transient hosts passed between workflow items as input/output might not work in all cases. Transient hosts rely on workflow cache, which doesn't work when asynchronous workflows are started, for example. Nested workflows might also fail.</li>
<li>Only GET and HEAD requests get redirected automatically. URL redirection uses the default strategy.</li>
<li>Host name verification is not supported.</li>
<li><em>Client certificate authentication is not supported.</em></li>
</ul>
</blockquote>
<p>I do not want to create persistent hosts in the plugin inventory as there may be a large number of clusters and this would lead a long list of host in the inventory, plus we have to make sure these hosts (with the certificates) are created and removed within the cluster lifecycle.</p>
<h2 id="downloading-the-certificates-with-python">Downloading the certificates with Python</h2>
<p>So let's switch to Python and see what needs to be done. In order to use client certificate authentication, we need to save the certificates into files and pass them to <tt>ssl</tt> module context.</p>
<p>In <a href="https://kuklis.github.io/cma/cma/post/vro8-polyglot-token">Getting token within vRO polyglot code</a> I described a method to acquire a token to use with Orchestrator/Automation. We can also gather Automation external URL from Orchestrator configuration so no need to provide it as an input. <em>Please note this will not work in a multitenant environment where the actual Automation URL contains the tenant name.</em> The Kubeconfig is in yaml format, but there is no yaml parsing module available by default in the Orchestrator implementation so we do a very basic string parsing to capture the certificates. We could import and place into a ZIP package the missing module but I wanted to keep the solution as simple as possible. Here is the code that prints the certificates within the Kubeconfig:</p>
<div class="highlight"><pre class="chroma"><code class="language-python" data-lang="python"><span class="ln"> 1</span><span class="kn">import</span> <span class="nn">json</span><span class="o">,</span> <span class="nn">http</span><span class="o">,</span> <span class="nn">ssl</span>
<span class="ln"> 2</span><span class="kn">from</span> <span class="nn">urllib.parse</span> <span class="kn">import</span> <span class="n">urlparse</span>
<span class="ln"> 3</span><span class="kn">from</span> <span class="nn">base64</span> <span class="kn">import</span> <span class="n">b64decode</span>
<span class="ln"> 4</span>
<span class="ln"> 5</span><span class="k">def</span> <span class="nf">handler</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">inputs</span><span class="p">):</span>
<span class="ln"> 6</span> <span class="n">jsonOut</span><span class="o">=</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span> <span class="n">separators</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;,&#39;</span><span class="p">,</span> <span class="s1">&#39;:&#39;</span><span class="p">))</span>
<span class="ln"> 7</span> <span class="k">print</span><span class="p">(</span><span class="s2">&#34;Inputs were {0}&#34;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">jsonOut</span><span class="p">))</span>
<span class="ln"> 8</span>
<span class="ln"> 9</span> <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;Authorization&#39;</span><span class="p">:</span> <span class="s2">&#34;Bearer &#34;</span> <span class="o">+</span> <span class="n">context</span><span class="p">[</span><span class="s2">&#34;getToken&#34;</span><span class="p">]()</span> <span class="p">}</span>
<span class="ln">10</span>
<span class="ln">11</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPConnection</span><span class="p">(</span><span class="s2">&#34;localhost&#34;</span><span class="p">,</span> <span class="mi">8280</span><span class="p">)</span>
<span class="ln">12</span> <span class="n">conn</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="s2">&#34;GET&#34;</span><span class="p">,</span> <span class="s2">&#34;/vco/api/server-configuration/settings&#34;</span><span class="p">,</span> <span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="n">headers</span><span class="p">)</span>
<span class="ln">13</span> <span class="n">res</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">getresponse</span><span class="p">()</span>
<span class="ln">14</span> <span class="k">print</span><span class="p">(</span><span class="s2">&#34;HTTP status: &#34;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">status</span><span class="p">))</span>
<span class="ln">15</span> <span class="n">data</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">read</span><span class="p">()</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s2">&#34;ascii&#34;</span><span class="p">)</span>
<span class="ln">16</span> <span class="k">print</span><span class="p">(</span><span class="s2">&#34;server configuration settings: &#34;</span> <span class="o">+</span> <span class="n">data</span><span class="p">)</span>
<span class="ln">17</span> <span class="n">settings</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="ln">18</span> <span class="n">hostname</span> <span class="o">=</span> <span class="n">urlparse</span><span class="p">(</span><span class="n">settings</span><span class="p">[</span><span class="s2">&#34;vco.csp.external.url&#34;</span><span class="p">])</span><span class="o">.</span><span class="n">hostname</span>
<span class="ln">19</span>
<span class="ln">20</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPSConnection</span><span class="p">(</span><span class="n">hostname</span><span class="p">,</span> <span class="n">context</span><span class="o">=</span><span class="n">ssl</span><span class="o">.</span><span class="n">SSLContext</span><span class="p">())</span>
<span class="ln">21</span> <span class="n">conn</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="s2">&#34;GET&#34;</span><span class="p">,</span> <span class="s2">&#34;/cmx/api/resources/k8s/clusters/&#34;</span> <span class="o">+</span> <span class="n">inputs</span><span class="p">[</span><span class="s2">&#34;clusterId&#34;</span><span class="p">]</span> <span class="o">+</span> <span class="s2">&#34;/kube-config&#34;</span><span class="p">,</span> <span class="s2">&#34;&#34;</span><span class="p">,</span> <span class="n">headers</span><span class="p">)</span>
<span class="ln">22</span> <span class="n">res</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">getresponse</span><span class="p">()</span>
<span class="ln">23</span> <span class="k">print</span><span class="p">(</span><span class="s2">&#34;HTTP status: &#34;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">status</span><span class="p">))</span>
<span class="ln">24</span> <span class="n">data</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">read</span><span class="p">()</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s2">&#34;ascii&#34;</span><span class="p">)</span>
<span class="ln">25</span> <span class="k">if</span> <span class="n">res</span><span class="o">.</span><span class="n">status</span> <span class="o">!=</span> <span class="mi">200</span><span class="p">:</span>
<span class="ln">26</span> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="s2">&#34;Could not fetch Kubeconfig of cluster with id &#34;</span> <span class="o">+</span> <span class="n">inputs</span><span class="p">[</span><span class="s2">&#34;clusterId&#34;</span><span class="p">]</span> <span class="o">+</span> <span class="s2">&#34;. &#34;</span> <span class="o">+</span> <span class="n">data</span><span class="p">)</span>
<span class="ln">27</span> <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">data</span><span class="o">.</span><span class="n">splitlines</span><span class="p">():</span>
<span class="ln">28</span> <span class="n">key</span><span class="p">,</span> <span class="n">sep</span><span class="p">,</span> <span class="n">value</span> <span class="o">=</span> <span class="n">line</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span><span class="o">.</span><span class="n">partition</span><span class="p">(</span><span class="s2">&#34;:&#34;</span><span class="p">)</span>
<span class="ln">29</span> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="s2">&#34;certificate-authority-data&#34;</span><span class="p">:</span>
<span class="ln">30</span> <span class="k">print</span><span class="p">(</span><span class="n">b64decode</span><span class="p">(</span><span class="n">value</span><span class="o">.</span><span class="n">strip</span><span class="p">(</span><span class="s1">&#39; &#34;&#39;</span><span class="p">))</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s2">&#34;ascii&#34;</span><span class="p">))</span>
<span class="ln">31</span> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="s2">&#34;client-certificate-data&#34;</span><span class="p">:</span>
<span class="ln">32</span> <span class="k">print</span><span class="p">(</span><span class="n">b64decode</span><span class="p">(</span><span class="n">value</span><span class="o">.</span><span class="n">strip</span><span class="p">(</span><span class="s1">&#39; &#34;&#39;</span><span class="p">))</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s2">&#34;ascii&#34;</span><span class="p">))</span>
<span class="ln">33</span> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="s2">&#34;client-key-data&#34;</span><span class="p">:</span>
<span class="ln">34</span> <span class="k">print</span><span class="p">(</span><span class="n">b64decode</span><span class="p">(</span><span class="n">value</span><span class="o">.</span><span class="n">strip</span><span class="p">(</span><span class="s1">&#39; &#34;&#39;</span><span class="p">))</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s2">&#34;ascii&#34;</span><span class="p">))</span>
<span class="ln">35</span> <span class="k">return</span> <span class="s2">&#34;done&#34;</span>
</code></pre></div><p>This code needs the ID of cluster within Automation, which we can get by Automation API:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span>$ curl -ksS -H <span class="s2">&#34;Authorization: Bearer </span><span class="nv">$access_token</span><span class="s2">&#34;</span> <span class="s1">&#39;https://vra8.corp.local/cmx/api/resources/k8s/clusters&#39;</span> <span class="p">|</span>
<span class="ln">2</span>&gt; jq -r <span class="s1">&#39;.content[]|[.id,.name]|@tsv&#39;</span>
<span class="ln">3</span>0f882cc4-8a84-4cf5-aa38-597e028d2a9d tst
<span class="ln">4</span>2fda385d-8b80-40e4-b277-be747804ff1a t3
<span class="ln">5</span>f9f61b48-a11d-4eb8-ae34-cd895af5c765 r2
<span class="ln">6</span>2f0faa48-cd89-4831-801f-b0e690c27465 t1