In Cloud Chats all data is encrypted in transit to and from the server with one key. Then the data is stored on the server encrypted with another key. Both those keys are known to the server. Secret Chats are encrypted with a key that is only known to you and your partner's devices. That's the main difference.
Now, security-wise there are four basic levels on which data can potentially be compromised.
Access to traffic: E.g. employer, ISP, sniffing hacker, person on the same public wifi.
Physical access to the data center: E.g.: state police, armed intruders, local engineers employed by Telegram.
Access to servers: The people who run the Telegram servers from their remote hideouts.
Access to your device: E.g. you, your mother, your spouse, corrupt police officer who took your phone.
Why Isn’t Telegram End-to-End encrypted by default? Pavel Durov explained it very well on this page.