General :: Windows - Code/software Access Locations In Memory That It's Not Supposed To?

By default, when applications run, they are limited in the amount of RAM they are allowed to access right? Technically my VLC could not access memory addresses used by Chrome. But how is it possible for the operating system/compiler to mistakingly allow access to an address a code is not allowed. I know alot of 'exploits' and 'payloads' take advantage of this fact and create variables that take up too much space and 'overflow' into other addresses, but how does this actually happen?

Are some programs more prone to this than others? Does it matter in which language they are coded in? For example, I know C is allowed to play around with memory, while Java is not. Also, what are the advantages of this? What if someone wrote malicious code to access someplace in memory, what could they do? The only thing I can think of is passwords/keys stored in RAM.

PS I thought about putting this in Stack Overflow, but my question is much broader than just specifically related to a programming perspective. If I've placed it in the wrong place, I'm sorry.