Monero is stored on an account, which is based on two distinct cryptographic keys: the spend key and the view key. Your spend key is the only key required to authorize the transfer of funds out of your Monero account. The view key grants access to view (but not spend) the balance of your account and can be handed over for the purposes of an audit.
Sample view key: de512ca7994295a90f5554bedc75612f5d5cd3dbdbc99c3dfa631c37eecec604
Sample spend key: 084afe9dfacb0012c7c553122300c42f0bbd1a5bfce86ac94f9c4e70fefd8b08
In addition to the spend and view keys, your Monero account also consists of a public address and mnemonic seed. A public address is what you share with others in order to be able to receive Monero. A mnemonic seed is a string of 13 or 25 words that you receive when you first create a new Monero account. This seed can be used to backup or restore your account, so it is absolutely imperative that you write it down and keep it in a safe place.
Sample mnemonic seed:
school exquisite owls maverick bounced pepper ruling oxidant scenic roster upbeat adept memoir onto wipeout surfer hubcaps agreed payment jagged aphid phone apricot awesome aphid
The most simple way to store your Monero is via the online web-wallet MyMonero, which works in your computer or mobile internet browser and doesn't require any installation. However, it is important to note that MyMonero's servers can see (but not spend) your Monero balance. Thus, you do run the slight risk of a loss of privacy. MyMonero is owned and operated by Riccardo Spagni, one of the Monero core team members.
If you would like to run a full Monero client while also contributing to the strength of the network, consider running a full Monero node. Doing so also affords you the highest level of privacy while still allowing quick access to your funds. It also does not require a large amount of processing power on your computer, though you will need a few gigabytes worth of disk space. Instructions (English only) on how to run a Monero node may be found here, and all the relevant downloads may be found here.
At the moment, there is no official GUI wallet for Monero, though one is currently in active development and will be released in 2016. For the time being, there are a number of 3rd party GUI wallets including LightWallet and MoneroX.
For absolute security, you may consider storing your Monero on a paper wallet, effectively moving the funds offline and into "cold storage." In order to remove your funds from the paper wallet and back online, you can either enter your mnemonic seed at MyMonero (which requires a 10 Monero fee) or by inputting the seed into your Monero client.
Note: As stated above, regardless of the method that you choose to store your Monero it is absolutely critical that you write down several copies of your mnemonic seed and store them in multiple safe places.