Sure, and that’s why it’s a regular expression - and not a string - replace.
In the given example you just need to escape the dot character to be considered as an actual dot.
i.e. [StringWithDots].RegexReplace("\.", "-").
As @Paul_Danyliuk suggests, double check with an online regexp reference.
Said that, for a full - and multiple - replacement, it is surely the most efficient way to achieve it.