Major hosted server stability issues

I bought a hosted Starbound server yesterday from Gameservers, 24 slots. Once we hit around 8-12 slots, things got really bad. Lag was a huge problem (it took several seconds for chat to be sent), and a lot of crashes. At one point, the server wouldn't get back up for another hour.

This morning, I bought a new 25 slot server from Nitrous. I am much happier with them; server performance is so much better. A very rare lag spike here or there, nothing too surprising. But again, once we get over 10 players, stability becomes a major issue. With Gameservers, the entire instance went down, and I'd have to force reboot the server. With Nitrous, these have just been server crashes; the console continues to run, and the server is back up in moments.

No mods have been used so far.

Thanks to Nitrous' better monitoring tools, I am able to get a better idea of what is going on. First, CPU and RAM usage graphs:



Every RAM spike is a crash. The highest point of CPU usage was a crash as well. It appears that the server crashes whenever it nearly reaches the max memory of my host. I've created a ticket asking for increased memory, but... shouldn't 4GB be enough?

I'll attach my logs. Most of the crashes appear to reference memory, but the last crash 9pm (logs are an hour behind, so 1800) look to be issues with world generation.

I want to get this to work, but my server community is getting frustrated, and I can't expect them to keep at it like this. I really hope to hear something soon about what is causing this.

edit: At midnight EST, my host gave us more memory. By 12:45, we were using 7.9GB of memory. We've only had one server crash, which looking at logs, was caused by a player leaving the Eucheris mission, trying to port to outpost, that failed so the game sent them to their ship, that failed which caused the crash.

Beyond that, it's been more stable, as the night went on, more laggy. We're seeing lag spikes, which correspond to this error in log: [22:094.066] [Error] ServerQueryThread exception caught: (NetworkException) udp recv error: A
--- Post updated ---
We need a dev response, because this is getting really bad. My server host:

 

Hello! This is being looked into - thank you for the report.

 

Just a quick +1 on this. I've got an ubuntu 16.04 virtual machine running on Hyper V, and I'd not enabled a maximum memory amount. I've just checked on it after this post, and it's running at 24GB, and doesn't really show any sign of slowing down in its increase.

This is with a maximum of 6 people on it at any one time. There is currently nobody on the server, and it's memory usage hasn't changed.

I'm happy to help the dev team investigate this, if there would be any useful information you'd like from my server.

 

Thank you, Molly!
--- Post updated ---
Maybe this will help: when my host had us at 20gb, we'd sit at that usage and the server was more or less stable, with some lag. Overnight they dropped us back down to 8 (originally paid for 4) and the server would reboot every few hours during US early morning whenever it hit 8gb usage.

 

Same problem here, i think garbage collector is not working properly...
Thanks Molly!!

 

Total server memory 8Gb. Got OOM after ~17 hours with 5-8 players. Noticed alot of lua related exceptions.

 

They fixed it with latest patch!

 

Thank you for the rapid fix! My server's been running for hours now, and it's at only 4GB RAM usage instead of 16GB + a ton of swap. It's awesome to know Chucklefish is listening.

 

Are things still running smoothly for everyone?
I'm trying to get friends into Starbound and want to rent a server for them - but if its still unstable it could scare them off (some are nubs)

 

Hi !

I'm self-hosting a server, and since 1.0.2 no more problems. It's stable and RAM usage is ok. I shut it down for a minute everyday to copy the storage folder and that's it.