Alternatively, rather than hold an action, you could preemptively do something that inhibits spellcasting that you expect to happen a certain way. Make a silent zone, to prevent incantations, or grapple the caster to prevent gestures, or create a dark cloud to block line of sight.