The original, initial idea of the WhatWG, of WebForms and HTML5 was, to make more things possible without Scripting. One reason was to make many basic functions available even when Scripting is off.
Of course, not everything would make sense to do without scripting, because this would get too complicated. But the use of the video- and audio-tag sure are one thing, that should be possible without scripting. I don't know, which technical changes have to be done to do this, but they sure should be done.