Notes.txt

Changed the java.security config file in order to remove "NULL" tls algorithm from the disabled tls algorithms
Otherwise, handshake will fail
to do so, redefine "jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon"
in the config/java.security file (notice there is no NULL)
NULL is for when there is no check I assume.
need to change for production


To create a replicated register, follow the steps below:
    1) create "hosts.config{#}" file under config/ directory. This file includes the list of hosts with their ip address and connection ports.
       It comes from the quorum of the object that can host it.
    2) create "system.config{#}" file under config/ directory similar to the existing file. The parameters are selfe explanatory.
        Added "system.servers.processNumber = #" entry to the system.config file
        It assigns a unique processNumber to each cluster. This is done to let multiple clusters run on same set of hosts if needed
        hosts.config{#} and currenView{#} are needed and created according to the {#}
    3) Create and run multiple instances of "RegisterServer..." with three arguments:
        first is the initial value of the register,
        second is the replica processNumber (must match the processNumber of the hosts.config and system.config),
        third is the cluster processNumber -> you need to keep track of cluster ids to the memory location each cluster stores
        normaly, each variable in the program must be stored in a replicated register. keep a mapping from cluster processNumber to
        variables.
    4) Create instances of RegisterClient...(clientID, clusterID) per program variable, you can call write(a,id) and read(id) on it


-For now, the method id and the method sequence number are passed to each method in the partition object
-Later think about how to move them to the runtime

make sure to set defaultkeys to false and usesignature to 1. also, in config*/keys there should exists a privatekey{#id} file

-ObjReCall rule which is implemented in register server and client by adding ids to client methods and a cache (map) from id to return value to the sever code

-Check why the ticket system use-case cannot get random user balance and ticket price?
(It return null when balance is returned randomly, but it works fine when it is a fixed number)


-To kill all the process running :
    kill $(lsof -t -i:13000-14000 -sTCP:LISTEN)

if faced error while issuing command over ssh without password (and not in the current shell) check all bellow:
    1) ssh-copy-id to the remote server
    2) permission of the public and private key are set correctly (https://www.golinuxcloud.com/postponed-publickey/)
    3) kinit user@REMOTE.HOST (echo 'password' | kinit fhous001@HPCC.UCR.EDU)
    4) the command path is in the $PATH environment variable of the remove server
        ~/.ssh/rc is called upon each connection to the remote
        do initialization of the $PATH, in this case:
        source /etc/profile.d/modules.sh



-Note that on the cluster (also on the local) all the nodes are reading/writing from/to one code base.
This caused problem when writing config files. now it is resolve by checking before writing to a new file


Possible bugs:
    1) resetting the object state after each call (remove for now and write false in OT use-case)
    2) blocking wait (busy while in invokeObj function). this throws NPE which causes problem. catching NPE in another while(true) results in wrong execution (why??) removed for now
    3) if at the begining of the execution bft library instances are not created, maybe it is because fields in the partitioned object are not accessible
    4) if getting connection refused or bind exception, check whether ports are already in use or not
    5) No appropriate protocol or cipher suite is disabled and,
        InvalidKeyStoreFormat exception:
            Java version must be greater that 11.0.150 (I use 14 local and java/11 on cluster)
    6) disabled cipher:
         add -Djdk.sunec.disableNative=false to the vm options
    7) In NodeClusterRunner servers are not starting (maybe because if statements are missing?)
    8) Make sure there is enough memory alloation in run.sh to each runtime (currently 1.5G) and consequently, in ClusterRunner

For building with aspectj with ant:
https://app.cs.amherst.edu/~jerager/cs23/doc/examples/build.xml


-To run on cluster:
  make sure you load java/11 module, otherwise "InvalidKeyStoreFormat" is thrown
  run "source compile.sh" to compile and package the source.
  run "java -cp bin/*:lib/* bftsmart.usecase.ClusterRunner {path_to_config} {number_of_repetitions}" to submit job
  make sure there are no errors in result.log


===========================
Fault injection hermes:
    make sure to set RUN inside ServerconnectionAspect
    commented BFTOrchestrationDaemon onAction faultContext.getRun() if condition value.
    for some reason the bft-library cannot handle crash in friendmap usecase (it crashes even with one crash)