webadmin stopped working on vmware.

[akito2001]

hermes webadmin stopped working. I get the login page but then a 502 gateway error after auth. Was working fine before and not sure what changed, havent ran any updates or anything Im aware of. Running in vmware. Seeing the below in tomcat logs, not sure if thats related. Any assistance would be much appreciated.

`14-May-2025 09:51:52.252 INFO [Thread-5] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["https-jsse-nio-8443"]
14-May-2025 09:51:52.275 INFO [Thread-5] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["https-jsse-nio-8443"]
14-May-2025 09:53:10.892 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-jsse-nio-8443"]
14-May-2025 09:53:11.387 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [864] milliseconds
14-May-2025 09:53:11.393 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
14-May-2025 09:53:11.393 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.31 (Ubuntu)]
14-May-2025 09:53:11.412 INFO [main] org.apache.catalina.startup.HostConfig.deployDescriptor Deploying deployment descriptor [/etc/tomcat9/Catalina/localhost/ciphermail.xml]

14-May-2025 09:53:11.438 WARNING [main] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of deployment descriptor [/etc/tomcat9/Catalina/localhost/ciphermail.xml] with an external docBase means the directory [/var/lib/tomcat9/webapps/ciphermail] in the appBase will be ignored

14-May-2025 09:53:16.375 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
14-May-2025 09:53:18.948 INFO [main] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of deployment descriptor [/etc/tomcat9/Catalina/localhost/ciphermail.xml] has finished in [7,536] ms
14-May-2025 09:53:18.948 INFO [main] org.apache.catalina.startup.HostConfig.deployDescriptor Deploying deployment descriptor [/etc/tomcat9/Catalina/localhost/web.xml]

14-May-2025 09:53:18.949 WARNING [main] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of deployment descriptor [/etc/tomcat9/Catalina/localhost/web.xml] with an external docBase means the directory [/var/lib/tomcat9/webapps/web] in the appBase will be ignored

14-May-2025 09:53:22.583 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
14-May-2025 09:53:24.427 INFO [main] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of deployment descriptor [/etc/tomcat9/Catalina/localhost/web.xml] has finished in [5,479] ms
14-May-2025 09:53:24.428 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/var/lib/tomcat9/webapps/ROOT]
14-May-2025 09:53:25.016 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
14-May-2025 09:53:25.019 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/var/lib/tomcat9/webapps/ROOT] has finished in [591] ms
14-May-2025 09:53:25.025 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-jsse-nio-8443"]
14-May-2025 09:53:25.041 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [13,653] milliseconds
`

[deeztek]

Hi,

this looks like an error for the ciphermail Web GUI. Have you tried restarting ciphermail back end:

systemctl restart djigzo

and then ciphermail front end:

systemctl restart tomcat9

[akito2001]

Ive rebooted the whole vm but ill give that a shot as well and let you know

[akito2001]

no change

[deeztek]

thanks for trying. Looking at it a little closer, the error log output you sent does not seem to indicate a problem. Can you check to ensure that your /etc/hosts has the following entry in it?

127.0.0.1 securemail.prosites.com

Thanks

[deeztek]

did you change the URL of your Hermes server recently?

[akito2001]

did you change the URL of your Hermes server recently?

no, we havent made any change recently, the url has remained the same since deployment (go live maybe a year ago). Also, I can get into ciphermail webui fine, just the hermes web admin is the issue. Ill check the hostfile thing

[akito2001]

[deeztek]

Okay that looks good.

ok that looks good. Can you check the hermesseg service:

systemctl status hermesseg

and maybe the following:

journalctl -xe | grep hermesseg

[deeztek]

The reason I asked about ciphermail was because the logs you sent looked like Tomcat logs from ciphermail.

[akito2001]

The reason I asked about ciphermail was because the logs you sent looked like Tomcat logs from ciphermail.

ya, I just put that there because I found it and wasnt sure if it was related. It looked like it might be an issue specifically around the app "main". Didnt know and was trying to provide as much detail as possible. Looking into the other service now

[akito2001]

[akito2001]

looks like its in a looping start failure. Now we are getting somewhere. Im not really seeing a reason though

[deeztek]

Can you post the contents of the following file:

/var/www/html/server.json

Thanks

[deeztek]

Also, please the following command in the cli as root:

# box

if you are asked to apply an update do NOT!

Once in the Commandbox prompt type the following and send me the output:

server info --verbose

Thanks

[akito2001]

Can you post the contents of the following file:

/var/www/html/server.json

Thanks

{
"name":"hermesseg",
"debug":false,
"trace":false,
"web":{
"aliases":{
"/admin":"/var/www/html/admin",
"/users":"/var/www/html/users",
"/main":"/var/www/html/main",
"/schedule":"/var/www/html/schedule",
"/user-auth":"/var/www/html/user-auth",
"/hermes-api":"/var/www/html/hermes-api"
},
"host":"0.0.0.0",
"HTTP":{
"enable":true,
"port":8888
},
"blockCFAdmin":"false",
"rewrites":{
"enable":"true"
},
"debug":{
"enable":"false"
}
},
"app":{
"sessionCookieSecure":true,
"sessionCookieHTTPOnly":true,
"openbrowser":false,
"debug":true,
"cfengine":"[email protected]+031",
"serverHomeDirectory":"/opt/CommandBox/web-contexts/wwwroot-hermesseg"
}
}

[akito2001]

Also, please the following command in the cli as root:

# box

if you are asked to apply an update do NOT!

Once in the Commandbox prompt type the following and send me the output:

server info --verbose

Thanks

[deeztek]

okay I think I see the issue. In the Commandbox prompt, enter the following and send me the output please:

server list

[akito2001]

server list

[deeztek]

The server's location should not be in root but rather in /opt/CommandBox/web-contexts/wwwroot-hermesseg. Not sure how that changed, but please try the following:

Stop the hermesseg service:

# systemctl stop hermesseg

Go into the Commandbox prompt:

# box

In the Commandbox prompt type the following and it will show you a list of servers in the stopped status:

CommandBox:wwwroot-hermesseg> server list
Processing (1) servers, please wait...

hermesseg (stopped)
  http://0.0.0.0:8888
  CF Engine: lucee 5.3.5+96
  Webroot: /var/www/html/
  Last Started: 15-May-2025 10:32:02

Next type the following command to forget the server and ensure you respond "Y" to confirm:

CommandBox:wwwroot-hermesseg> server forget hermesseg
Really forget & delete server 'hermesseg' forever [y/n]? : y
Poof! Wiped out server hermesseg
CommandBox:wwwroot-hermesseg>

Next exit the Commandbox prompt:

exit

Next start the hermesseg service:

systemctl start hermesseg

Let me know how that works.

[akito2001]

Ill give that a shot today and let you know

[akito2001]

That looks like it worked. The web admin is back up now.

[deeztek]

Glad you got it working.