- Gateway address would not actually change if it was not deleted
- If gateway was deleted and recreated, then you would need to use different public IP address
The second scenario could happen if, for example, you needed to change gateway type from Policy-based to Route-based.
Now there is an option to use static public IP addresses for Azure VNET Gateways.
This option is available for new VpnGw1AZ, VpnGw2AZ and VpnGw3AZ SKUs.