Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infeasibility more cleaning #2231

Merged
merged 18 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
0580371
Renaming infeasibility analyzer : renaming in class Constraint
guilpier-code Jul 2, 2024
e46bc1f
Renaming infeasibility analyzer : renaming in class InfeasibleProblem…
guilpier-code Jul 2, 2024
eec4b4f
Simplifying infeasibility analyzer
guilpier-code Jul 2, 2024
f10c53a
format
flomnes Jul 3, 2024
b4f47bd
Merge remote-tracking branch 'github/develop' into fix/simplifying-in…
flomnes Jul 3, 2024
644335b
New constraint to detect by infeasibility analyzer
guilpier-code Jul 2, 2024
b94dfc0
Infeasibility - to more changeability : simplification + renaming
guilpier-code Jul 3, 2024
f408ecd
Infeasibility - to more changeability : simplifying function StringBe…
guilpier-code Jul 3, 2024
bb55301
Infeasibility - to more changeability : simplifying functions areaNam…
guilpier-code Jul 3, 2024
1d58402
Infeasibility - to more changeability : renaming Constraint::getType(…
guilpier-code Jul 3, 2024
666201d
Infeasibility - to more changeability : function STSName() ==> simpli…
guilpier-code Jul 3, 2024
499619b
Infeasibility - to more changeability : remove unnecessary comments
guilpier-code Jul 3, 2024
25d3424
Infeasibility - to more changeability : simplifying function sortCons…
guilpier-code Jul 3, 2024
3c040c0
Infeasibility - to more changeability : in unit tests, change mislead…
guilpier-code Jul 3, 2024
b3fc6f0
Infeasibility - to more changeability : split constraints logs into 2…
guilpier-code Jul 3, 2024
bbdc230
Infeasibility - to more changeability : move call to sortConstraintsB…
guilpier-code Jul 3, 2024
331c1f5
Infeasibility - to more changeability : run clang-format
guilpier-code Jul 4, 2024
20603f7
Merge remote-tracking branch 'github/develop' into fix/infeasibility-…
flomnes Jul 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 41 additions & 101 deletions src/solver/infeasible-problem-analysis/constraint.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,40 +25,26 @@
#include <iomanip>
#include <sstream>

#include <boost/algorithm/string.hpp>
#include <boost/algorithm/string/regex.hpp>
#include <boost/regex.hpp>

namespace
{
const std::string kUnknown = "<unknown>";
}

namespace Antares::Optimization
{
Constraint::Constraint(const std::string& input, const double slackValue):
name_(input),
Constraint::Constraint(const std::string& name, const double slackValue):
name_(name),
slackValue_(slackValue)
{
}

std::size_t Constraint::extractComponentsFromName()
void Constraint::extractComponentsFromName()
{
const auto beg = name_.begin();
const auto end = name_.end();
std::size_t newPos = 0;
const std::size_t sepSize = 2;
const std::size_t inputSize = name_.size();
for (std::size_t pos = 0; pos < inputSize; pos = newPos + sepSize)
{
newPos = name_.find("::", pos);
if (newPos == std::string::npos)
{
nameComponents_.emplace_back(beg + pos, end);
break;
}
if (newPos > pos)
{
nameComponents_.emplace_back(beg + pos, beg + newPos);
}
}
return nameComponents_.size();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to return the size : this was returned only to test if this size is zero, but it can never be zero, as constraints are filtered based on regex pattern :

const std::string constraint_name_pattern = "^AreaHydroLevel::|::hourly::|::daily::|::weekly::|"
                                                "^FictiveLoads::|^Level::|"
                                                "^HydroPower::";

This forces the constraint's name to contain "::" and something else (like AreaHydroLevel, daily, ...).

boost::algorithm::split_regex(nameComponents_, name_, boost::regex("::"));
}

double Constraint::getSlackValue() const
Expand All @@ -75,67 +61,36 @@ class StringIsNotWellFormated: public std::runtime_error
}
};

std::string StringBetweenAngleBrackets(const std::string& str)
std::string StringBetweenAngleBrackets(const std::string& constraintName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function is agnostic to its input and therefore the param should not be renamed;

{
const auto& begin = str.begin();
const auto& end = str.end();
std::vector<std::string> split_name;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we base the extraction of the string between angle bracket on the split algorithm, rather than the find algorithm. It makes code slightly more simple.
Error handling was a bit simplified.
In the end, function is shorter and easier to understand, I guess.

boost::split(split_name, constraintName, boost::is_any_of("<>"));

auto left = std::find(begin, end, '<');

if (left == end)
std::string err_msg = "Error: ";
if (split_name.size() < 3)
{
std::ostringstream stream;
stream << std::string("Error the string: ") << std::quoted(str)
<< " does not contains the left angle bracket " << std::quoted("<");
throw StringIsNotWellFormated(stream.str());
err_msg += "constraint name '" + constraintName + "' misses '<' and/or '>' bracket";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or make it a class method

throw StringIsNotWellFormated(err_msg);
}

auto right = std::find(begin, end, '>');
if (right == end)
if (split_name[1].empty())
{
std::ostringstream stream;
stream << std::string("Error the string: ") << std::quoted(str)
<< " does not contains the right angle bracket " << std::quoted(">");
throw StringIsNotWellFormated(stream.str());
err_msg += "constraint name '" + constraintName + "' must be of format '*<str>*'";
throw StringIsNotWellFormated(err_msg);
}

if (std::distance(left, right) <= 1)
{
std::ostringstream stream;
stream << std::string("Error the string: ") << std::quoted(str) << " must be of format "
<< std::quoted("*<str>*");
throw StringIsNotWellFormated(stream.str());
}
return std::string(left + 1, right);
return split_name[1];
}

std::string Constraint::getAreaName() const
std::string Constraint::areaName() const
{
if ((getType() == ConstraintType::binding_constraint_hourly)
|| (getType() == ConstraintType::binding_constraint_daily)
|| (getType() == ConstraintType::binding_constraint_weekly))
{
return "<none>";
}
return StringBetweenAngleBrackets(nameComponents_.at(1));
}

std::string Constraint::getTimeStepInYear() const
std::string Constraint::timeStep() const
{
switch (getType())
{
case ConstraintType::binding_constraint_hourly:
case ConstraintType::binding_constraint_daily:
case ConstraintType::fictitious_load:
case ConstraintType::hydro_reservoir_level:
case ConstraintType::short_term_storage_level:
return StringBetweenAngleBrackets(nameComponents_.at(nameComponents_.size() - 2));
default:
return kUnknown;
}
return StringBetweenAngleBrackets(nameComponents_.at(nameComponents_.size() - 2));
}

ConstraintType Constraint::getType() const
ConstraintType Constraint::type() const
{
assert(nameComponents_.size() > 1);
if (nameComponents_.at(1) == "hourly")
Expand All @@ -158,61 +113,46 @@ ConstraintType Constraint::getType() const
{
return ConstraintType::hydro_reservoir_level;
}
if (nameComponents_.at(0) == "HydroPower")
Copy link
Contributor

@a-zakir a-zakir Jul 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's works because Antares provides constraints and vars names, trimmed!

{
return ConstraintType::hydro_production_weekly;
}
if (nameComponents_.at(0) == "Level")
{
return ConstraintType::short_term_storage_level;
}
return ConstraintType::none;
}

std::string Constraint::getBindingConstraintName() const
std::string Constraint::shortName() const
{
switch (getType())
{
case ConstraintType::binding_constraint_hourly:
case ConstraintType::binding_constraint_daily:
case ConstraintType::binding_constraint_weekly:
return nameComponents_.at(0);
default:
return kUnknown;
}
return nameComponents_.at(0);
}

std::string Constraint::getSTSName() const
std::string Constraint::STSname() const
{
if (getType() == ConstraintType::short_term_storage_level)
{
return StringBetweenAngleBrackets(nameComponents_.at(2));
}
else
{
return kUnknown;
}
return StringBetweenAngleBrackets(nameComponents_.at(2));
}

std::string Constraint::prettyPrint() const
{
switch (getType())
switch (type())
{
case ConstraintType::binding_constraint_hourly:
return "Hourly binding constraint '" + getBindingConstraintName() + "' at hour "
+ getTimeStepInYear();
return "Hourly binding constraint '" + shortName() + "' at hour " + timeStep();
case ConstraintType::binding_constraint_daily:
return "Daily binding constraint '" + getBindingConstraintName() + "' at day "
+ getTimeStepInYear();
return "Daily binding constraint '" + shortName() + "' at day " + timeStep();
case ConstraintType::binding_constraint_weekly:
return "Weekly binding constraint '" + getBindingConstraintName();

return "Weekly binding constraint '" + shortName();
case ConstraintType::fictitious_load:
return "Last resort shedding status at area '" + getAreaName() + "' at hour "
+ getTimeStepInYear();
return "Last resort shedding status at area '" + areaName() + "' at hour " + timeStep();
case ConstraintType::hydro_reservoir_level:
return "Hydro reservoir constraint at area '" + getAreaName() + "' at hour "
+ getTimeStepInYear();
return "Hydro reservoir constraint at area '" + areaName() + "' at hour " + timeStep();
case ConstraintType::hydro_production_weekly:
return "Hydro weekly production at area '" + areaName() + "'";
case ConstraintType::short_term_storage_level:
return "Short-term-storage reservoir constraint at area '" + getAreaName() + "' in STS '"
+ getSTSName() + "' at hour " + getTimeStepInYear();

return "Short-term-storage reservoir constraint at area '" + areaName() + "' in STS '"
+ STSname() + "' at hour " + timeStep();
default:
return kUnknown;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ class ConstraintSlackAnalysis: public UnfeasibilityAnalysis

std::vector<const operations_research::MPVariable*> slackVariables_;
const std::string constraint_name_pattern = "^AreaHydroLevel::|::hourly::|::daily::|::weekly::|"
"^FictiveLoads::|^Level::";
"^FictiveLoads::|^Level::|"
"^HydroPower::";
};

} // namespace Antares::Optimization
Original file line number Diff line number Diff line change
Expand Up @@ -32,34 +32,31 @@ enum class ConstraintType
binding_constraint_weekly,
fictitious_load,
hydro_reservoir_level,
hydro_production_weekly,
short_term_storage_level,
none
};

class Constraint
{
public:
// Construct object
flomnes marked this conversation as resolved.
Show resolved Hide resolved
Constraint() = default;
Constraint(const std::string& input, const double slackValue);
Constraint(const std::string& name, const double slackValue);

// Raw members
double getSlackValue() const;

// Extract items, check consistency
std::size_t extractComponentsFromName();
void extractComponentsFromName();
std::string prettyPrint() const;
ConstraintType getType() const;
ConstraintType type() const;

private:
std::string name_;
std::vector<std::string> nameComponents_;
double slackValue_;

// Get specific items
std::string getAreaName() const;
std::string getSTSName() const;
std::string getTimeStepInYear() const;
std::string getBindingConstraintName() const;
std::string areaName() const;
std::string STSname() const;
std::string timeStep() const;
std::string shortName() const;
};
} // namespace Antares::Optimization
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,11 @@ class InfeasibleProblemReport
private:
void turnSlackVarsIntoConstraints(
const std::vector<const operations_research::MPVariable*>& slackVariables);
void sortConstraints();
void sortConstraintsBySlackValue();
void trimConstraints();
void sortConstraintsByType();
void logSuspiciousConstraints();
void logInfeasibilityCauses();

std::vector<Constraint> constraints_;
std::map<ConstraintType, unsigned int> nbConstraintsByType_;
Expand Down
30 changes: 16 additions & 14 deletions src/solver/infeasible-problem-analysis/report.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,9 @@ InfeasibleProblemReport::InfeasibleProblemReport(
const std::vector<const MPVariable*>& slackVariables)
{
turnSlackVarsIntoConstraints(slackVariables);
sortConstraints();
sortConstraintsBySlackValue();
trimConstraints();
sortConstraintsByType();
}

void InfeasibleProblemReport::turnSlackVarsIntoConstraints(
Expand All @@ -56,28 +57,23 @@ void InfeasibleProblemReport::turnSlackVarsIntoConstraints(
}
}

void InfeasibleProblemReport::sortConstraints()
void InfeasibleProblemReport::sortConstraintsBySlackValue()
{
std::sort(std::begin(constraints_), std::end(constraints_), ::compareSlackSolutions);
}

void InfeasibleProblemReport::trimConstraints()
{
if (nbMaxVariables <= constraints_.size())
{
constraints_.resize(nbMaxVariables);
}
unsigned int nbConstraints = constraints_.size();
constraints_.resize(std::min(nbMaxVariables, nbConstraints));
}

void InfeasibleProblemReport::sortConstraintsByType()
{
for (auto& c: constraints_)
{
if (c.extractComponentsFromName() == 0)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As already said above, extractComponentsFromName was returning the number of components a constraint's name is composed with, but this number can never be zero.
So the if statement was removed.

{
return;
}
nbConstraintsByType_[c.getType()]++;
c.extractComponentsFromName();
nbConstraintsByType_[c.type()]++;
}
}

Expand All @@ -88,12 +84,20 @@ void InfeasibleProblemReport::logSuspiciousConstraints()
{
Antares::logs.error() << c.prettyPrint();
}
}

void InfeasibleProblemReport::logInfeasibilityCauses()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we split logs into 2 separate functions :

  • one that logs suspicious constraints
  • one that logs of infeasibility causes

{
Antares::logs.error() << "Possible causes of infeasibility:";
if (nbConstraintsByType_[ConstraintType::hydro_reservoir_level] > 0)
{
Antares::logs.error() << "* Hydro reservoir impossible to manage with cumulative options "
"\"hard bounds without heuristic\"";
}
if (nbConstraintsByType_[ConstraintType::hydro_production_weekly] > 0)
{
Antares::logs.error() << "* impossible to generate exactly the weekly hydro target";
}
if (nbConstraintsByType_[ConstraintType::fictitious_load] > 0)
{
Antares::logs.error() << "* Last resort shedding status,";
Expand All @@ -112,14 +116,12 @@ void InfeasibleProblemReport::logSuspiciousConstraints()
{
Antares::logs.error() << "* Binding constraints,";
}

Antares::logs.error() << "* Negative hurdle costs on lines with infinite capacity (rare).";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't see any use for this log line :
it's printed in all cases of infeasibility, even if here are no link involved (indeed, hurdle costs are related to links).
Besides, it can be misleading for user, forced to wonder why he's got this message.

}

void InfeasibleProblemReport::prettyPrint()
{
sortConstraintsByType();
Copy link
Contributor Author

@guilpier-code guilpier-code Jul 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to sortConstraintsByType() was moved above, with another call to a sort function.
Besides, this call is not related to client function's name, which is supposed to print only.

logSuspiciousConstraints();
logInfeasibilityCauses();
}

} // namespace Antares::Optimization
Original file line number Diff line number Diff line change
Expand Up @@ -171,9 +171,9 @@ std::unique_ptr<MPSolver> createUnfeasibleProblem(const std::string& constraintN
}

static const std::string validConstraintNames[] = {
"BC::hourly::hour<36>",
"BC::daily::day<67>",
"BC::weekly::week<12>",
"BC-name-1::hourly::hour<36>",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I changed a bit the binding constraints' names.
Originally, these names started with "BC::", and it made think that BC names in mps files necessarily start with this prefix. This not the case, this prefix is the name of the BC.

"BC-name-2::daily::day<67>",
"BC-name-3::weekly::week<12>",
"FictiveLoads::hour<25>",
"AreaHydroLevel::hour<8>",
};
Expand Down
Loading