Well, you mentioned your site, so I assume you mean a web site (other types of signing/encryption/key exchange are similiar, but I just want to focus on web for you). Your web site creates public/private key pairs - they MUST be created as pairs, btw. In general, a client retrieves the public key from your site and uses it for encryption. Not gonna go into symmetric/asymmetric keys in detail, just know that you end up creating a cipher text (symmetric key, using the public key that was sent from server to client) that gets sent to the server, and then the server uses the private key to decrypt that data (key). Now you both (client and server) have a symmetric key that is only known to you both, and that is then used to encrypt the communication between the two.
A certificate is information about the server, and the public key itself. Getting that 'signed' by a CA (Certificate Authority) validates that the public key is indeed valid for the site in question. Basically, a server creates a key and provides it to the CA with their information, the CA makes damn sure things are on the up and up, and then creates their OWN private key to pair with the public key for YOUR server - they keep that. They send the 'signed' certificate back to you. Browsers/OS's know who the CA's are, so you get a cert (key) from your server (on the client), and it then validates it by talking to the CA - since they have the private key of their own for that public key, then it will only pass if it's valid. Now your client knows the server didn't somehow get hijacked with some trick like DNS redirection or something. Once that's done, your system will put it in the 'Certificate Store' to keep, so it doesn't have to talk to the CA all the time. That lets you talk to the site, while already having the proper public key for that site in place and validated.
Hope that's not too muddy. I probably left something out, but that's the gist of it.
(Edit, for summary):
1) Client connects to server, over secure connection - requesting one.
2) Server sends public key to client.
3) If the cert is not in your local Certificate Store on the client, the client uses CA to validate the public key (if not, warning/etc. is issued)
4) If CA validates the cert (public key and info), it is added to your local Certificate Store.
4) Client uses the public key to create a new symmetric key, encrypts that with the (validated) public key, and sends that to server.
5) Server uses its own private key to decrypt that, and that now has the same temporary key you created, so now both ends have that temporary key (the symmetric key).
6) Server and client talk together encrypting the data using the shared symmetric key.
For Certificate(s):
1) Server applies for certificate from CA (data such as public key, organization, contact, etc.).
2) CA encrypts that with it their own private key - they create a document (X.509) that is digitally signed using that private key.
3) CA sends that certificate back to the server
4) Server installs the certificate
htw
