Is there a way to remove the chat button in the widget?

No, but you can hide the menu option using the query parameters, then the chat will also be disabled.

You can add your feature request here