How to accept any unknown options

[dfleury2]

Hi,
I am trying a use CLI11 configuration file where I do not know which options are available. A kind a pass through to another lib.
For example:

[kafka]
topic = TOPIC_NAME

[kafka.options]
A = 360
B = /path/to/something
C = list of some values
X = another

Here, I got a "kafka" command, that accept topic as an option (a fairly common one for kafka),
and I try to add an "options" subcommand that will accept any options (A, B, C, ...) with its associated values.

What will be the best way to acheive this (if possible), without using "options" becoming an option mapped to vector of string and keep the .ini format for this subcommand.
So not like this

[kafka]
topic = TOPIC_NAME

options = ["A = 360",
    "B = /path/to/something",
    "C = list of some values",
    "X = another"]

So far, I have tried options subcommand with nameless option without any success (like all my previous attempt, allow extra flag, ...).

Latest experimentation, with drawbacks (allow config extra in app is not wanted but only at options subcommand level).
I do not see the value in the remaining loop.

#include <CLI/CLI.hpp>

#include <iostream>
#include <vector>
#include <string>

using Options = std::vector<std::string>;

int main(int argc, char **argv) {
    std::string topic_name;
    Options options;

    CLI::App app;
    app.configurable()->allow_config_extras(true); // want false here
    app.set_config("-c")->check(CLI::ExistingFile);

    auto cmd = app.add_subcommand("kafka");
    cmd->add_option("--topic", topic_name);

    auto kafka_options = cmd->add_subcommand("options");
    kafka_options->allow_config_extras(true);

    CLI11_PARSE(app, argc, argv);

    std::cout << "Remaining: \n";
    for (const auto &o: kafka_options->remaining()) std::cout << o << std::endl;
}

[phlptp]

One thing that might help is

CLI::App app;
    app.set_config("-c")->check(CLI::ExistingFile);
    app.get_config_formatter_base()->section("kafka");
    app.add_option("--topic", topic_name);
    auto kafka_options = app.add_subcommand("options");
    kafka_options->allow_config_extras(true);

This should limit the configuration parsing to the specific section and ignore everything outside of it. I know I use this for parsing when the options are part of a much bigger configuration file.

[phlptp]

Ok, I was trying to figure out why that wouldn't work for you. There is an example in the configFileTest.cpp if you are interested

[dfleury2]

Greatly appreciate your help. I will try to figure out how I can solve this. Thanks.

[dfleury2]

Ok, If I am not wrong, I was not clear, (I will re-read your answer tomorrow), but
If I use this program:

#include <CLI/CLI.hpp>

#include <iostream>
#include <string>

int main(int argc, char **argv) {
    std::string program_name;
    std::string topic_name;
    bool auto_commit{false};

    CLI::App app;
    app.set_config("-c")->check(CLI::ExistingFile);
    app.configurable(true);
    app.allow_config_extras(true);

    auto prog = app.add_subcommand("main");
    prog->add_option("--program", program_name);

    auto kafka = app.add_subcommand("kafka");
    kafka->add_option("--topic", topic_name);
    kafka->add_option("--auto-commit", auto_commit);

    auto kafka_options = kafka->add_subcommand("options");
    kafka_options->allow_config_extras(true);

    CLI11_PARSE(app, argc, argv);

    std::cout << "Program name: " << program_name << std::endl;
    std::cout << "Topic name: " << topic_name << std::endl;
    std::cout << "Auto Commit: " << auto_commit << std::endl;
    std::cout << "Remaining:\n";
    for (const auto& o: kafka_options->remaining()) {
        std::cout << "  " << o << std::endl;
    }
}

wih this config file

[main]
program = Pair

[kafka]
topic = TopicName
auto-commit = true

[kafka.options]
A = 360
B = /path/to/something
C = list of some values
X = another
Y = again
Z = probably not the last one

I got this output:

Program name: Pair
Topic name: TopicName
Auto Commit: 1
Remaining:
  kafka.options.A
  kafka.options.B
  kafka.options.C
  kafka.options.X
  kafka.options.Y
  kafka.options.Z

I just want to have access to the ConfigItem.inputs of the remaining. But stil not see how to get them, they are only store in the missing_ var.
And I do not want to exclude the other commands from parsing.

PS: I wonder if app.allow_config_extras(true);is not a bug since I only want extra conf for kafka.options subcommand.

[phlptp]

Not 100% sure I know what you are asking? Are you wanting the unrecognized options and subsequent values from the config file stored in missing_ structure? I don't think we have a way of doing that currently. Could probably add one without too much effort, but it hasn't come up before.

I think in most of the use cases in the past that was either used as a warning mechanism to identify which parts of the config file weren't recognized or we just wanted them ignored completely.

[dfleury2]

Yes, I need a way to get these "unprocessed" values so I can move them to external libs without knowing what they are exactly.
For example Kafka client has a lot of options available and I do not want to "register" them in CLI11 but only to have access to the key, value pair so I can send them to the cppkafka lib for example.

[dfleury2]

My first attempt, is something like that;

It will give me access to this in the remaining:

Program name: Pair
Topic name: TopicName
Auto Commit: 1
Remaining for kafka.options:
   A = 360
   B = /path/to/something
   C = list of some values
   X = another
   Y = again
   Z = probably not the last one

But probably not the best way to integrate this in CLI11 (I remove the full_name since this is not what I expect from it. I.e. at kafka.options level I do not want "kafka.options.A" in the name).

Another way could be to create a missing callback that will be called with the item (but this willl expose some internal struct).

Another could be to support "*" option name so any option can be support (no more missing), and used with a callback. More complex and could break thing somewhere else.