Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

In KairosDB query proxy, when KairosDB errors, error doesn't propagate right to MP response #329

Open
speezepearson opened this issue Dec 12, 2019 · 0 comments
Open

In KairosDB query proxy, when KairosDB errors, error doesn't propagate right to MP response #329

speezepearson opened this issue Dec 12, 2019 · 0 comments

Comments

@speezepearson
Copy link
Contributor

speezepearson commented Dec 12, 2019 •
edited
Loading

Steps to reproduce:

  • curl --verbose 'http://localhost:8080/api/v1/datapoints/query' -H 'content-type: application/json' --data-binary '{"cache_time":0,"start_absolute":1576087117000,"end_absolute":1576108717000,"metrics":[{"limit":0,"name":"web_perf/time_to_interactive","tags":{"browser_name":["Firefox"],"region":["Europe"]},"aggregators":[{"name":"rate","align_sampling":true,"align_start_time":true,"unit":"HOURS"}]}]}' --compressed

    (the crucial bit is the rate aggregator)

  • see in the logs:

    {
  "time": "2019-12-11T23:42:47.382Z",
  "name": "log",
  "level": "crit",
  "data": {
    "message": "error processing request",
    "request": {
      "_id": "77316300",
      "_class": "play.mvc.Http$RequestImpl"
    }
  },
  "exception": {
    "type": "java.util.concurrent.CompletionException",
    "message": "com.arpnetworking.kairos.client.KairosDbRequestException: {\"errors\":[\"Property 'align_start_time' was specified for object 'rate' but no matching setter was found on 'class org.kairosdb.core.aggregator.RateAggregator'\"]}",
    "backtrace": [
      "at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)",
      "at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)",
      "at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:604)",
      "at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)",
      "at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)",
      "at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)",
      "at scala.concurrent.java8.FuturesConvertersImpl$CF.apply(FutureConvertersImpl.scala:20)",
      "at scala.concurrent.java8.FuturesConvertersImpl$CF.apply(FutureConvertersImpl.scala:18)",
      "at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)",
      "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:63)",
      "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:78)",
      "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)",
      "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)",
      "at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)",
      "at scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:54)",
      "at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)",
      "at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:106)",
      "at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)",
      "at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)",
      "at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)",
      "at scala.concurrent.Promise$class.complete(Promise.scala:55)",
      "at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:157)",
      "at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)",
      "at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)",
      "at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)",
      "at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)",
      "at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)",
      "at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:81)",
      "at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)",
      "at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:80)",
      "at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)",
      "at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)",
      "at scala.concurrent.Promise$class.trySuccess(Promise.scala:94)",
      "at scala.concurrent.impl.Promise$DefaultPromise.trySuccess(Promise.scala:157)",
      "at akka.stream.impl.HeadOptionStage$$anon$3.onPush(Sinks.scala:241)",
      "at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:519)",
      "at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:508)",
      "at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:378)",
      "at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:588)",
      "at akka.stream.impl.fusing.GraphInterpreterShell.init(ActorGraphInterpreter.scala:558)",
      "at akka.stream.impl.fusing.ActorGraphInterpreter.tryInit(ActorGraphInterpreter.scala:679)",
      "at akka.stream.impl.fusing.ActorGraphInterpreter.preStart(ActorGraphInterpreter.scala:727)",
      "at akka.actor.Actor$class.aroundPreStart(Actor.scala:528)",
      "at akka.stream.impl.fusing.ActorGraphInterpreter.aroundPreStart(ActorGraphInterpreter.scala:670)",
      "at akka.actor.ActorCell.create(ActorCell.scala:656)",
      "at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:527)",
      "at akka.actor.ActorCell.systemInvoke(ActorCell.scala:549)",
      "at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:283)",
      "at akka.dispatch.Mailbox.run(Mailbox.scala:224)",
      "at akka.dispatch.Mailbox.exec(Mailbox.scala:235)",
      "at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)",
      "at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)",
      "at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)",
      "at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)"
    ],
    "data": {
      "_id": "4709353a",
      "_class": "java.util.concurrent.CompletionException",
      "cause": {
        "type": "com.arpnetworking.kairos.client.KairosDbRequestException",
        "message": "{\"errors\":[\"Property 'align_start_time' was specified for object 'rate' but no matching setter was found on 'class org.kairosdb.core.aggregator.RateAggregator'\"]}",
        "backtrace": [
          "at com.arpnetworking.kairos.client.KairosDbClientImpl.lambda$2(KairosDbClientImpl.java:138)",
          "at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)",
          "at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)",
          "at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)",
          "at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)",
          "at scala.concurrent.java8.FuturesConvertersImpl$CF.apply(FutureConvertersImpl.scala:20)",
          "at scala.concurrent.java8.FuturesConvertersImpl$CF.apply(FutureConvertersImpl.scala:18)",
          "at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)",
          "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:63)",
          "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:78)",
          "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)",
          "at scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)",
          "at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)",
          "at scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:54)",
          "at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)",
          "at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:106)",
          "at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)",
          "at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)",
          "at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)",
          "at scala.concurrent.Promise$class.complete(Promise.scala:55)",
          "at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:157)",
          "at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)",
          "at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)",
          "at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)",
          "at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)",
          "at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)",
          "at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:81)",
          "at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)",
          "at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:80)",
          "at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)",
          "at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)",
          "at scala.concurrent.Promise$class.trySuccess(Promise.scala:94)",
          "at scala.concurrent.impl.Promise$DefaultPromise.trySuccess(Promise.scala:157)",
          "at akka.stream.impl.HeadOptionStage$$anon$3.onPush(Sinks.scala:241)",
          "at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:519)",
          "at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:508)",
          "at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:378)",
          "at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:588)",
          "at akka.stream.impl.fusing.GraphInterpreterShell.init(ActorGraphInterpreter.scala:558)",
          "at akka.stream.impl.fusing.ActorGraphInterpreter.tryInit(ActorGraphInterpreter.scala:679)",
          "at akka.stream.impl.fusing.ActorGraphInterpreter.preStart(ActorGraphInterpreter.scala:727)",
          "at akka.actor.Actor$class.aroundPreStart(Actor.scala:528)",
          "at akka.stream.impl.fusing.ActorGraphInterpreter.aroundPreStart(ActorGraphInterpreter.scala:670)",
          "at akka.actor.ActorCell.create(ActorCell.scala:656)",
          "at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:527)",
          "at akka.actor.ActorCell.systemInvoke(ActorCell.scala:549)",
          "at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:283)",
          "at akka.dispatch.Mailbox.run(Mailbox.scala:224)",
          "at akka.dispatch.Mailbox.exec(Mailbox.scala:235)",
          "at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)",
          "at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)",
          "at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)",
          "at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)"
        ],
        "data": {
          "_id": "1b3eeb24",
          "_class": "com.arpnetworking.kairos.client.KairosDbRequestException",
          "httpStatus": 400,
          "httpMessage": "Bad Request",
          "requestUri": "http://localhost:9415/api/v1/datapoints/query"
        }
      }
    }
  },
  "context": {
    "host": "<snip>",
    "processId": "27868",
    "threadId": "mportal-akka.actor.default-dispatcher-2",
    "logger": "g.ErrorHandler",
    "line": "50",
    "file": "ErrorHandler.java",
    "class": "global.ErrorHandler"
  },
  "id": "f4ea0f7d-0fa2-4c78-a7b6-f255c91c3bfb",
  "version": "0"
}

Relevant code: https://github.com/ArpNetworking/metrics-portal/blob/master/app/com/arpnetworking/kairos/client/KairosDbClientImpl.java#L138
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@speezepearson