The rotary dial on the back only dictates the HDMI and SDI output resolution. You would want to adjust the settings in the IP interface of the camera to set the output of the IP feed. Just type the IP address of the unit into a browser to get into the webgui (admin for username and password) and go to the video section. The first stream sets the ethernet output settings. You can see what to set them to according to the guide below for best results. Be sure to reboot the camera after changing and applying any settings.
That all sounds like a good setup. If the video is coming in choppy it may be due to the computer or the USB chipset being overloaded. The HDMI output at 1080p30 into a Magewell should be fine. Is OBS running with good CPU usage? Is it possible to go to 1080p60 (dial setting 0) as this should alleviate any kind of jittery video. If not, please feel free to put in a ticket at PTZOptics.com and we can go over some settings and I can potential remote into your machine if needed to take a look at things.
Are you able to open the feed in any other applications (AMCAP, VLC, Zoom.US) and see the video or is this issue presenting itself everywhere?
Have you been able to verify that the Magewell Adapter is working properly by connecting an HDMI source, other than the camera, to it?
In the OSD of the camera the only setting that could possibly impact the HDMI feed would be in the following location:
Navigate to DVIMode
Set to HDMI for YUV Encoding (default)
Set to DVI for RGB Encoding
We look forward to hearing any feedback from the above suggestions in hopes of getting to the root of your issue.
Very glad to hear that everything is working as you expect now.
If you ever have problems in the future please don't hesitate to return to the forums or reach out to our support team.