Skip to content

Commit

Permalink
Update documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@actions-user
actions-user committed Aug 15, 2023
1 parent 9600ae0 commit a5250ec
Show file tree
Hide file tree
Showing 137 changed files with 24,865 additions and 7 deletions.
2,351 changes: 2,351 additions & 0 deletions genindex.html
View file

Large diffs are not rendered by default.

Binary file modified objects.inv
View file
Binary file not shown.
53 changes: 51 additions & 2 deletions parts/callbacks.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,57 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">

<div class="section" id="callbacks">
<h1>Callbacks<a class="headerlink" href="#callbacks" title="Permalink to this headline"></a></h1>
<div class="section" id="module-gym_electric_motor.callbacks">
<span id="callbacks"></span><h1>Callbacks<a class="headerlink" href="#module-gym_electric_motor.callbacks" title="Permalink to this headline"></a></h1>
<p>This module introduces predefined callbacks for the GEM environment.</p>
<dl class="class">
<dt id="gym_electric_motor.callbacks.RampingLimitMargin">
<em class="property">class </em><code class="sig-prename descclassname">gym_electric_motor.callbacks.</code><code class="sig-name descname">RampingLimitMargin</code><span class="sig-paren">(</span><em class="sig-param">initial_limit_margin=(-0.1</em>, <em class="sig-param">0.1)</em>, <em class="sig-param">maximum_limit_margin=(-1</em>, <em class="sig-param">1)</em>, <em class="sig-param">step_size=0.1</em>, <em class="sig-param">update_time='episode'</em>, <em class="sig-param">update_freq=10</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/callbacks.html#RampingLimitMargin"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.callbacks.RampingLimitMargin" title="Permalink to this definition"></a></dt>
<dd><p>Callback used to adapt the limit margin of a reference generator during runtime.</p>
<p>Supports all
<a class="reference internal" href="reference_generators/subepisoded_reference_generator.html#gym_electric_motor.reference_generators.subepisoded_reference_generator.SubepisodedReferenceGenerator" title="gym_electric_motor.reference_generators.subepisoded_reference_generator.SubepisodedReferenceGenerator"><code class="xref py py-mod docutils literal notranslate"><span class="pre">SubepisodedReferenceGenerator</span></code></a>
and <code class="xref py py-mod docutils literal notranslate"><span class="pre">SwitchedReferenceGenerator</span></code>
with only
<a class="reference internal" href="reference_generators/subepisoded_reference_generator.html#gym_electric_motor.reference_generators.subepisoded_reference_generator.SubepisodedReferenceGenerator" title="gym_electric_motor.reference_generators.subepisoded_reference_generator.SubepisodedReferenceGenerator"><code class="xref py py-mod docutils literal notranslate"><span class="pre">SubepisodedReferenceGenerator</span></code></a>
as sub generators.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>initial_limit_margin</strong> (<em>tuple</em><em>(</em><em>floats</em><em>)</em>) – The initial limit margin which gets updated by AdaptiveLimitMargin
until it reaches maximum_limit_margin</p></li>
<li><p><strong>maximum_limit_margin</strong> (<em>tuple</em><em>(</em><em>floats</em><em>)</em>) – The maximum limit margin. This will be the limit margin after
AdaptiveLimitMargin’s last update</p></li>
<li><p><strong>step_size</strong> (<em>float</em>) – The value by which each limit gets updated at each step</p></li>
<li><p><strong>update_time</strong> (<em>string</em>) – When the update happens. “step” for the end of a step, “episode”
for the end of an episode</p></li>
<li><p><strong>update_freq</strong> (<em>int</em>) – After how many cumulative units of update_time an update occurs</p></li>
</ul>
</dd>
</dl>
<dl class="simple">
<dt>Additional Notes:</dt><dd><p>All limit_margins should be between -1 and 1</p>
</dd>
</dl>
<dl class="method">
<dt id="gym_electric_motor.callbacks.RampingLimitMargin.on_reset_end">
<code class="sig-name descname">on_reset_end</code><span class="sig-paren">(</span><em class="sig-param">state</em>, <em class="sig-param">reference</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/callbacks.html#RampingLimitMargin.on_reset_end"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.callbacks.RampingLimitMargin.on_reset_end" title="Permalink to this definition"></a></dt>
<dd><p>Gets called at the end of each reset</p>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.callbacks.RampingLimitMargin.on_step_end">
<code class="sig-name descname">on_step_end</code><span class="sig-paren">(</span><em class="sig-param">k</em>, <em class="sig-param">state</em>, <em class="sig-param">reference</em>, <em class="sig-param">reward</em>, <em class="sig-param">terminated</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/callbacks.html#RampingLimitMargin.on_step_end"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.callbacks.RampingLimitMargin.on_step_end" title="Permalink to this definition"></a></dt>
<dd><p>Gets called at the end of each step</p>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.callbacks.RampingLimitMargin.set_env">
<code class="sig-name descname">set_env</code><span class="sig-paren">(</span><em class="sig-param">env</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/callbacks.html#RampingLimitMargin.set_env"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.callbacks.RampingLimitMargin.set_env" title="Permalink to this definition"></a></dt>
<dd><p>Sets the environment of the motor.</p>
</dd></dl>

</dd></dl>

</div>


Expand Down
91 changes: 91 additions & 0 deletions parts/constraint_monitor.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,97 @@ <h2>Usage Guide<a class="headerlink" href="#usage-guide" title="Permalink to thi
</div>
<div class="section" id="constraint-monitor-api-documentation">
<h2>Constraint Monitor API Documentation<a class="headerlink" href="#constraint-monitor-api-documentation" title="Permalink to this headline"></a></h2>
<dl class="class">
<dt id="gym_electric_motor.core.ConstraintMonitor">
<em class="property">class </em><code class="sig-prename descclassname">gym_electric_motor.core.</code><code class="sig-name descname">ConstraintMonitor</code><span class="sig-paren">(</span><em class="sig-param">limit_constraints=()</em>, <em class="sig-param">additional_constraints=()</em>, <em class="sig-param">merge_violations='max'</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/core.html#ConstraintMonitor"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.ConstraintMonitor" title="Permalink to this definition"></a></dt>
<dd><p>The ConstraintMonitor is used within the ElectricMotorEnvironment to monitor the states for illegal / undesired
values (e.g. overcurrents).</p>
<p>It consists of a list of multiple independent constraints. Each constraint gets the current observation of the
environment as input and returns a <em>violation degree</em> within <span class="math notranslate nohighlight">\([0.0, 1.0]\)</span>.
All these are merged together and the ConstraintMonitor returns a total violation degree.</p>
<dl class="simple">
<dt><strong>Soft Constraints:</strong></dt><dd><p>To enable a higher flexibility, the constraints return a violation degree (float) instead of a simple violation
flag (bool). So, even before the limits are violated, the reward function can take the limit violation degree
into account. If the violation degree is at 0.0, no states are in a dangerous region. For values between 0.0 and
1.0 the reward will be decreased gradually so that the agent will learn to avoid these state regions.
If the violation degree reaches 1.0 the episode is terminated.</p>
</dd>
<dt><strong>Hard Constraints:</strong></dt><dd><p>With the above concept, also hard constraints that directly terminate an episode without any “danger”-region
can be modeled. Then, the violation degree of the constraint directly changes from 0.0 to 1.0, if a violation
occurs.</p>
</dd>
</dl>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>limit_constraints</strong> (<em>list</em><em>(</em><em>str</em><em>)</em><em>/'all_states'</em>) – <dl class="simple">
<dt>Shortcut parameter to pass all states that limits shall be observed.</dt><dd><ul>
<li><dl class="simple">
<dt>list(str): Pass a list with state_names and all of the states will be observed to stay within</dt><dd><p>their limits.</p>
</dd>
</dl>
</li>
<li><p>’all_states’: Shortcut for all states are observed to stay within the limits.</p></li>
</ul>
</dd>
</dl>
</p></li>
<li><p><strong>additional_constraints</strong> (<em>list</em><em>(</em><em>Constraint/callable</em><em>)</em>) – Further constraints that shall be monitored. These have to be initialized first and passed to the
ConstraintMonitor. Alternatively, constraints can be defined as a function that takes the current
state and returns a float within [0.0, 1.0].</p></li>
<li><p><strong>merge_violations</strong> (‘max’/’product’/callable(<a href="#id3"><span class="problematic" id="id4">*</span></a>violation_degrees) -&gt; float) – <p>Function to merge all single
violation degrees to a total violation degree.</p>
<blockquote>
<div><ul>
<li><p>’max’: Take the maximal violation degree as total violation degree.</p></li>
<li><dl class="simple">
<dt>’product’: Calculates the total violation degree as one minus the product of one minus all single</dt><dd><p>violation degrees.</p>
</dd>
</dl>
</li>
<li><p>callable(<a href="#id1"><span class="problematic" id="id2">*</span></a>violation_degrees) -&gt; float: User defined function to calculate the total violation.</p></li>
</ul>
</div></blockquote>
</p></li>
</ul>
</dd>
</dl>
<dl class="method">
<dt id="gym_electric_motor.core.ConstraintMonitor.check_constraints">
<code class="sig-name descname">check_constraints</code><span class="sig-paren">(</span><em class="sig-param">state: numpy.ndarray</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/core.html#ConstraintMonitor.check_constraints"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.ConstraintMonitor.check_constraints" title="Permalink to this definition"></a></dt>
<dd><p>Function to check and merge all constraints.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><p><strong>state</strong> (<em>ndarray</em><em>(</em><em>float</em><em>)</em>) – The current environments state.</p>
</dd>
<dt class="field-even">Returns</dt>
<dd class="field-even"><p>The total violation degree in [0,1]</p>
</dd>
<dt class="field-odd">Return type</dt>
<dd class="field-odd"><p>float</p>
</dd>
</dl>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.core.ConstraintMonitor.constraints">
<em class="property">property </em><code class="sig-name descname">constraints</code><a class="headerlink" href="#gym_electric_motor.core.ConstraintMonitor.constraints" title="Permalink to this definition"></a></dt>
<dd><p>Returns the list of all constraints the ConstraintMonitor observes.</p>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.core.ConstraintMonitor.set_modules">
<code class="sig-name descname">set_modules</code><span class="sig-paren">(</span><em class="sig-param">ps: gym_electric_motor.core.PhysicalSystem</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/gym_electric_motor/core.html#ConstraintMonitor.set_modules"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.ConstraintMonitor.set_modules" title="Permalink to this definition"></a></dt>
<dd><p>The PhysicalSystem of the environment is passed to save important parameters like the index of the states.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><p><strong>ps</strong> (<a class="reference internal" href="physical_systems/physical_system.html#gym_electric_motor.core.PhysicalSystem" title="gym_electric_motor.core.PhysicalSystem"><em>PhysicalSystem</em></a>) – The PhysicalSystem of the environment.</p>
</dd>
</dl>
</dd></dl>

</dd></dl>

</div>
</div>

Expand Down
90 changes: 90 additions & 0 deletions parts/constraints/constraint.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,96 @@ <h2>How To: Define Your Own Constraints<a class="headerlink" href="#how-to-defin
</div>
<div class="section" id="constraint-api-documentation">
<h2>Constraint API Documentation<a class="headerlink" href="#constraint-api-documentation" title="Permalink to this headline"></a></h2>
<dl class="class">
<dt id="gym_electric_motor.core.RewardFunction">
<em class="property">class </em><code class="sig-prename descclassname">gym_electric_motor.core.</code><code class="sig-name descname">RewardFunction</code><a class="reference internal" href="../../_modules/gym_electric_motor/core.html#RewardFunction"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.RewardFunction" title="Permalink to this definition"></a></dt>
<dd><p>The abstract base class for reward functions in gym electric motor environments.</p>
<p>The reward function is called once per step and returns reward for the current time step.</p>
<dl class="attribute">
<dt id="gym_electric_motor.core.RewardFunction.reward_range">
<code class="sig-name descname">reward_range</code><a class="headerlink" href="#gym_electric_motor.core.RewardFunction.reward_range" title="Permalink to this definition"></a></dt>
<dd><p>Defining lowest and highest possible rewards.</p>
<dl class="field-list simple">
<dt class="field-odd">Type</dt>
<dd class="field-odd"><p>Tuple(float, float)</p>
</dd>
</dl>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.core.RewardFunction.close">
<code class="sig-name descname">close</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../_modules/gym_electric_motor/core.html#RewardFunction.close"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.RewardFunction.close" title="Permalink to this definition"></a></dt>
<dd><p>Called, when the environment is closed to store logs, close files etc.</p>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.core.RewardFunction.reset">
<code class="sig-name descname">reset</code><span class="sig-paren">(</span><em class="sig-param">initial_state=None</em>, <em class="sig-param">initial_reference=None</em><span class="sig-paren">)</span><a class="reference internal" href="../../_modules/gym_electric_motor/core.html#RewardFunction.reset"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.RewardFunction.reset" title="Permalink to this definition"></a></dt>
<dd><p>This function is called by the environment when reset.</p>
<p>Inner states of the reward function can be reset here, if necessary.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>initial_state</strong> (<em>ndarray</em><em>(</em><em>float</em><em>)</em>) – Initial state array of the Environment</p></li>
<li><p><strong>initial_reference</strong> (<em>ndarray</em><em>(</em><em>float</em><em>)</em>) – Initial reference array of the environment.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.core.RewardFunction.reward">
<code class="sig-name descname">reward</code><span class="sig-paren">(</span><em class="sig-param">state</em>, <em class="sig-param">reference</em>, <em class="sig-param">k=None</em>, <em class="sig-param">action=None</em>, <em class="sig-param">violation_degree=0.0</em><span class="sig-paren">)</span><a class="reference internal" href="../../_modules/gym_electric_motor/core.html#RewardFunction.reward"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.RewardFunction.reward" title="Permalink to this definition"></a></dt>
<dd><p>Reward calculation. If limits have been violated the reward is calculated with a separate function.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>state</strong> (<em>ndarray</em><em>(</em><em>float</em><em>)</em>) – Environments state array.</p></li>
<li><p><strong>reference</strong> (<em>ndarray</em><em>(</em><em>float</em><em>)</em>) – Environments reference array.</p></li>
<li><p><strong>k</strong> (<em>int</em>) – Systems momentary time-step</p></li>
<li><p><strong>action</strong> (<em>element of action space</em>) – The previously taken action.</p></li>
<li><p><strong>violation_degree</strong> (<em>float in</em><em> [</em><em>0.0</em><em>, </em><em>1.0</em><em>]</em>) – Degree of violation of the constraints. 0.0 indicates that all
constraints are complied. 1.0 indicates that the constraints have been so much violated, that a reset is
necessary.</p></li>
</ul>
</dd>
<dt class="field-even">Returns</dt>
<dd class="field-even"><p>Reward for this state, reference, action tuple.</p>
</dd>
<dt class="field-odd">Return type</dt>
<dd class="field-odd"><p>float</p>
</dd>
</dl>
</dd></dl>

<dl class="attribute">
<dt>
<code class="sig-name descname">reward_range</code><em class="property"> = (-inf, inf)</em></dt>
<dd><p>Lower and upper possible reward</p>
<dl class="field-list simple">
<dt class="field-odd">Type</dt>
<dd class="field-odd"><p>Tuple(int,int)</p>
</dd>
</dl>
</dd></dl>

<dl class="method">
<dt id="gym_electric_motor.core.RewardFunction.set_modules">
<code class="sig-name descname">set_modules</code><span class="sig-paren">(</span><em class="sig-param">physical_system</em>, <em class="sig-param">reference_generator</em>, <em class="sig-param">constraint_monitor</em><span class="sig-paren">)</span><a class="reference internal" href="../../_modules/gym_electric_motor/core.html#RewardFunction.set_modules"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#gym_electric_motor.core.RewardFunction.set_modules" title="Permalink to this definition"></a></dt>
<dd><p>Setting of the physical system, to set state arrays fitting to the environments states</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>physical_system</strong> (<a class="reference internal" href="../physical_systems/physical_system.html#gym_electric_motor.core.PhysicalSystem" title="gym_electric_motor.core.PhysicalSystem"><em>PhysicalSystem</em></a>) – The physical system of the environment</p></li>
<li><p><strong>reference_generator</strong> (<a class="reference internal" href="../reference_generators/reference_generator.html#gym_electric_motor.core.ReferenceGenerator" title="gym_electric_motor.core.ReferenceGenerator"><em>ReferenceGenerator</em></a>) – The reference generator of the environment.</p></li>
<li><p><strong>constraint_monitor</strong> (<a class="reference internal" href="../core.html#gym_electric_motor.core.ConstraintMonitor" title="gym_electric_motor.core.ConstraintMonitor"><em>ConstraintMonitor</em></a>) – The constraint monitor of the environment.</p></li>
</ul>
</dd>
</dl>
</dd></dl>

</dd></dl>

</div>
</div>

Expand Down
Loading

0 comments on commit a5250ec

Please sign in to comment.