How to install the latest version of Covalent from pip?

[santoshkumarradha]

Hey @AgnostiqHQ team,

I am trying to install the latest version of covalent from pip (v0.89.2) but running into some weird bugs/errors. I suspect it might be because of clashes with my previous covalent installation. Could someone let me know how exactly to cleanly install the new version ?

TIA !

[Emmanuel289]

@santoshkumarradha These cmds should generally work for a fresh install:

• killall python
• covalent purge
• pip uninstall cova
• pip install cova
• pip list | grep cova
• covalent start

[santoshkumarradha]

Thanks @Emmanuel289 ! quick question, do I need to do anything to my old config file and such ? I see new config files as well as .env files. (hopefully I done need to touch them ?)

[Emmanuel289]

You're welcome! @santoshkumarradha. As far as I am aware, the new config and .env files do not need to be modified after a clean install and workflows should proceed as expected. Are you trying to execute some specific action?

[santoshkumarradha]

I was just trying to understand if the old config file is automatically deleted with either pip uninstall or if the current one overrides it or something of those sorts. But If I don't need to tinker with config file, then I shall not !

[santoshkumarradha]

Note: I was getting multiple errors for some reason. Tried purging, killinfgsupervrisor, killing individual things, nothing seems to take care, one thing or the other ended up in a fatal state.

Supervisord is running in process 53798.
covalent:data                     STARTING  
covalent:dispatcher               STARTING  
covalent:dispatcher_mq_consumer   RUNNING   pid 53809, uptime 0:00:08
covalent:nats                     BACKOFF   Exited too quickly (process log may have details)
covalent:queuer                   STARTING  
covalent:results                  STARTING  
covalent:runner                   STARTING  
covalent:ui                       BACKOFF   Exited too quickly (process log may have details)

But following the procedure that @Emmanuel289 put out to fresh install whenever i run into server problems seems to make covalent back up and running.

[santoshkumarradha]

Okay sorry, I was quick to jump on this. I still see NATS as fatal

Supervisord is running in process 54439.
covalent:data                     STOPPED   Apr 23 12:20 PM
covalent:dispatcher               STOPPED   Apr 23 12:20 PM
covalent:dispatcher_mq_consumer   STOPPED   Apr 23 12:20 PM
covalent:nats                     FATAL     Exited too quickly (process log may have details)
covalent:queuer                   STOPPED   Apr 23 12:20 PM
covalent:results                  STOPPED   Apr 23 12:20 PM
covalent:runner                   STOPPED   Apr 23 12:20 PM
covalent:ui                       STOPPED   Apr 23 12:20 PM

even after reinstalling with the above procedure.

[Emmanuel289]

@santoshkumarradha This seems to be an unusual behavior from nats-server. It's likely that the nats-server was already started...Did you try checking if the nats-server is running on port 4222? Then killing the process and restarting.

[santoshkumarradha]

Okay let me try that.

[AlejandroEsquivel]

@santoshkumarradha Currently I am investigating ways to ensure microservices boot up robustly on install in cases where covalent wasn’t brought down gracefully on previous installs.

If you could run the following script and provide the output that might give more insight into what could be going wrong:

import psutil
import re
import socket

from subprocess import Popen

def port_from_pid(pid: int):
    if psutil.pid_exists(pid):
        connections = psutil.Process(pid).connections()
        if len(connections) == 0:
            return None
        else:
            return connections[0].laddr.port
    return None

def is_port_in_use(port: int, host: str = "localhost") -> bool:
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        return sock.connect_ex((host, port)) == 0

def enumerate_conflicting_pid_by_cmd():
    # not assuming only one process running
    supervisord_pids = []
    nats_pids = []

    # aggregate pids based on command used to invoke process
    for process in psutil.process_iter():
        try:
            # we want to identify commands of the form:
            # 1. python(2/3/..) .../bin/supervisord ...
            # 2. python(2/3...) .../bin/nats-server.py
            cmd = process.cmdline()
            if len(cmd) >= 2:
                pid = process.pid
                main_process_cmd = cmd[0].split('/')[-1]
                first_arg = cmd[1].split('/')[-1]
                is_python_process = re.search(r"^python([0-9](\.)?[0-9]{0,3})?$",main_process_cmd) is not None
                is_supervisord = is_python_process and first_arg.lower() == 'supervisord'
                is_nats_server = is_python_process and first_arg.lower() == 'nats-server'
                if is_supervisord: 
                    print(f'Supervisord pid: {pid} - Port: {port_from_pid(pid)}')
                    supervisord_pids.append(pid)
                    
                if is_nats_server:
                    print(f'NATS pid: {pid} - Port: {port_from_pid(pid) or 4222}')
                    nats_pids.append(pid)
        except (psutil.AccessDenied, psutil.NoSuchProcess) as e:
            # there may be permission errors getting cmdline() of certain processes, we ignore these
            pass
   
    return supervisord_pids, nats_pids


def enumerate_conflicting_app_processes_by_port():

    cova_port_set = [8000,8001,8002,8003,8004,8005,4222,9001]

    for port in cova_port_set:
        is_in_use = is_port_in_use(port)
        if is_in_use:
            print(f"Port: {port} occupied")


enumerate_conflicting_app_processes_by_port()
enumerate_conflicting_pid_by_cmd()