Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Charts improvements and bug fixes #164

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Charts improvements and bug fixes #164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
45c8415
add debug options and NPE fixes
iosdev747 Sep 16, 2024
4f390b1
keep app configs in all members
iosdev747 Sep 16, 2024
0e44230
add debug flags adding ports and java args
iosdev747 Sep 16, 2024
f528ae0
add env vars for jvm args and debug mode
iosdev747 Sep 18, 2024
d7499d3
Merge remote-tracking branch 'upstream/master' into charts_improvements
iosdev747 Oct 12, 2024
4fc3c8a
add otlp configs in helm charts
iosdev747 Oct 12, 2024
508336d
port related minor refactors
iosdev747 Oct 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .run/varadhi local [run].run.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@
<RunAsTest>false</RunAsTest>
<method v="2" />
</configuration>
</component>
</component>
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ OR

Start [varadhi local \[run\]](.run%2Fvaradhi%20local%20%5Brun%5D.run.xml) IntelliJ run profile.

## k8s Deployment

```helm install varadhi-server setup/helm/varadhi -f setup/helm/varadhi/values/server_values.yaml```
```helm install varadhi-controller setup/helm/varadhi -f setup/helm/varadhi/values/controller_values.yaml```

## Modules

- entities: It has all the entities used by the spi module and varadhi apis.
Expand Down
2 changes: 2 additions & 0 deletions server/src/main/java/com/flipkart/varadhi/config/MemberConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,6 @@ public class MemberConfig {
private int clusterPort;
private int maxQps;
private int networkMBps;
private int cpuCount;
private int nicMBps;
}
Copy link
Collaborator

@kmrdhruv kmrdhruv Sep 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead we can adjust the .values file to have new config.

13 changes: 10 additions & 3 deletions setup/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# base image for this should be jre based on instead of jdk.
# Ubuntu 22.04.2 LTS
# Ubuntu 22.04.2 LTS
FROM eclipse-temurin:17.0.7_7-jdk
ARG SKIP_CERT_CHECK=false
ARG VARADHI_HOME=.

ENV PORT=18488
Expand All @@ -19,14 +20,20 @@ RUN setcap cap_net_raw,cap_net_admin=eip /usr/bin/ngrep
# Install Zookeeper client.
RUN mkdir -p /tools/zk
WORKDIR /tools/zk
RUN wget https://archive.apache.org/dist/zookeeper/zookeeper-3.6.0/apache-zookeeper-3.6.0-bin.tar.gz
RUN if "$SKIP_CERT_CHECK"; \
then wget https://archive.apache.org/dist/zookeeper/zookeeper-3.6.0/apache-zookeeper-3.6.0-bin.tar.gz --no-check-certificate ; \
kmrdhruv marked this conversation as resolved.
Show resolved Hide resolved
else wget https://archive.apache.org/dist/zookeeper/zookeeper-3.6.0/apache-zookeeper-3.6.0-bin.tar.gz ; \
fi
RUN tar -xf apache-zookeeper-3.6.0-bin.tar.gz
RUN rm apache-zookeeper-3.6.0-bin.tar.gz

# Install Pulsar Client.
RUN mkdir -p /tools/pulsar
WORKDIR /tools/pulsar
RUN wget https://archive.apache.org/dist/pulsar/pulsar-3.0.0/apache-pulsar-shell-3.0.0-bin.tar.gz
RUN if "$SKIP_CERT_CHECK"; \
then wget https://archive.apache.org/dist/pulsar/pulsar-3.0.0/apache-pulsar-shell-3.0.0-bin.tar.gz --no-check-certificate ; \
else wget https://archive.apache.org/dist/pulsar/pulsar-3.0.0/apache-pulsar-shell-3.0.0-bin.tar.gz ; \
fi
RUN tar -xf apache-pulsar-shell-3.0.0-bin.tar.gz
RUN rm apache-pulsar-shell-3.0.0-bin.tar.gz

Expand Down
9 changes: 8 additions & 1 deletion setup/docker/scripts/entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,11 @@ JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.ssl=false"
JAVA_OPTS="$JAVA_OPTS -Dlog4j2.configurationFile=/etc/varadhi/log4j2.xml"

exec java -cp ./*:dependencies/* $JAVA_OPTS com.flipkart.varadhi.VaradhiApplication /etc/varadhi/configuration.yml
# If debug flag is set(1 or true) then enable remote debugging
if [[ ! -z "$APP_DEBUG" ]]; then
if [[ "$APP_DEBUG" == "1" || "$APP_DEBUG" == "true" ]]; then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any issue being it just set to any value ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, idea was if someone sets to false or 0 or any other invalid value, we don't enable it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why two levels of if block needed, only inner block should suffice ?

JAVA_OPTS="$JAVA_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:5005"
fi
fi

exec java -cp ./*:dependencies/* $JAVA_OPTS $JAVA_EXTRA_OPTS com.flipkart.varadhi.VaradhiApplication /etc/varadhi/configuration.yml
12 changes: 11 additions & 1 deletion setup/helm/varadhi/templates/_server.tpl → setup/helm/varadhi/templates/_app.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{/*
Varadhi Server configuration.
*/}}
{{- define "configMap.varadhi.server" -}}
{{- define "configMap.varadhi.app" -}}

{{ with .Values.varadhi.app.member }}
member:
Expand Down Expand Up @@ -65,4 +65,14 @@ featureFlags:

{{ template "configMap.metastore.zookeeper" . }}

{{ with .Values.varadhi.app.controller }}
controller:
{{- toYaml . | nindent 2 }}
{{- end }}

{{ with .Values.otlpConfig }}
otlpConfig:
{{- toYaml . | nindent 2 }}
{{- end }}

{{- end }}
4 changes: 1 addition & 3 deletions setup/helm/varadhi/templates/app-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,4 @@ metadata:
component: {{ .Values.deployment.name }}
data:
configuration.yml: |
{{- if eq .Values.deployment.name "server" }}
{{ include "configMap.varadhi.server" . | nindent 4 }}
{{- end }}
{{ include "configMap.varadhi.app" . | nindent 4 }}
16 changes: 16 additions & 0 deletions setup/helm/varadhi/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,22 @@ spec:
{{- end }}
resources:
{{- toYaml .Values.deployment.resources | nindent 12 }}
env:
- name: APP_DEBUG
value: "{{ .Values.varadhi.app.debug }}"
- name: JAVA_EXTRA_OPTS
value: "{{ .Values.varadhi.app.configs.javaExtraOpts }}"
- name: OTEL_SERVICE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
ports:
- containerPort: {{ .Values.varadhi.app.ports.jmx }} # jmx remote port
name: jmx
{{- if .Values.varadhi.app.debug }}
- containerPort: {{ .Values.varadhi.app.ports.jvmDebug }} # jvm remote Debug port
name: jvm-debug
{{- end }}
volumeMounts:
- mountPath: /etc/varadhi
name: configs
Expand Down
6 changes: 6 additions & 0 deletions setup/helm/varadhi/templates/server-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ spec:
targetPort: {{ .Values.varadhi.app.httpServerOptions.port }}
protocol: TCP
name: server-http
{{- if .Values.varadhi.app.debug }}
- port: {{ .Values.varadhi.app.ports.jvmDebug }}
targetPort: {{ .Values.varadhi.app.ports.jvmDebug }}
protocol: TCP
name: jvm-debug
{{- end }}
selector:
{{- include "varadhi.commonLabels" . | nindent 4 }}
component: {{ .Values.deployment.name }}
Expand Down
25 changes: 23 additions & 2 deletions setup/helm/varadhi/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,11 @@ images:
####################################################
varadhi:
app:
configs:
javaExtraOpts: "-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.rmi.port=9990"
name: server
configMapName: server
debug: false
restOptions:
deployedRegion: "default"
defaultOrg: "default"
Expand Down Expand Up @@ -182,10 +185,19 @@ varadhi:
providerClassName: "com.flipkart.varadhi.db.ZookeeperProvider"
featureFlags:
leanDeployment: false
controller:
maxConcurrentOps: 2
maxRetryAllowed: 3
retryIntervalInSeconds: 10
retryMinBackoffInSeconds: 10
retryMaxBackOffInSeconds: 60
member:
roles: [ "Server", "Controller" ]
cpuCount: 1
nicMBps: 100
ports:
jmx: 9990
jvmDebug: 5005

#############################################
# Varadhi Server logging config
Expand All @@ -212,12 +224,12 @@ messaging:
configMapName: messaging
pulsar:
adminOptions:
serviceHttpUrl: "http://192.168.1.3:8080"
serviceHttpUrl: "http://host.docker.internal:8080"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be time to decide the hierarchy of values files. Suggestion/TBD : values -> fcp.values -> role specific i.e (controller|consumer|server|standalone) and override only required content in higher level files ?

connectionTimeoutMs: 2000
readTimeoutMs: 2000
requestTimeoutMs: 2000
clientOptions:
serviceUrl: "http://192.168.1.3:8080"
serviceUrl: "http://host.docker.internal:8080"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create standalone values file to override this, as value is specific to the standalone setup.

keepAliveIntervalSecs: 30 # Pulsar default (30 secs)
ioThreads: 2 # start with 2, might need further tuning (Pulsar default is 1).
connectionsPerBroker: 1 # Pulsar default is 1.
Expand All @@ -238,3 +250,12 @@ messaging:
authzProvider:
configMapName: authorization

#############################################
# OTEL config
#############################################
otlpConfig:
otlp.url: "http://host.docker.internal:4318/v1/metrics"
otlp.step: "20s"
otlp.aggregationTemporality: "CUMULATIVE"
otlp.resourceAttributes:
otlp.headers:
Loading
Loading