A private app is the same kind of app the Marketplace lists, built on ZAF and installed only in your account. It can sit in the ticket sidebar, the new-ticket sidebar, the user or organization pages, the top bar or the nav bar, or run in the background with no panel at all.
Credentials for the other system go in secure settings, so they're added to requests on Zendesk's side and never exposed in the agent's browser. That's what lets one panel read an ERP, a carrier and an accounting system together, and offer buttons that act on them.
Developers package and upload apps with Zendesk's command-line tools, and can try them in a test account before anything reaches your agents. The Zendesk API reference covers the REST side these apps usually lean on.