I have looked at options in List Management but do not see any option there to add a header (h2, h3...).
This will be fixed in our Bootstrap 4 upgrade where each of the headers has an appropriate h3 tag on any page with a header title. Believe you can enforce this from Joomla it self using menu parameters though depending on your Joomla template used. If you're talking about the column headers no those won't have a header tag as they've been fixed with the appropriate role attribute.
I realize this could be done using a field that is a text editor area, but we don't want users formatting fields on their own.
Is there any plan to take a look into accessibility issues?
Yes, we intend to completely redesign our userlists at some point. Just no idea when. Everything else we try to maintain role and aria attributes where necessary as best as possible. We can't really guarantee screen reader compatibility though given there's so many. Admittedly accessibility checks for the blind are not something I perform as I simply don't have the time.