There are 2 types of players in your friends' bar: friends and neighbors. Friends are added automatically from your Facebook friend list. Neighbors are K&B players who you add by GAME IDs.
To add neighbors, click the "Neighbors" button on the bottom panel.
You can add up to 300 neighbors. Incoming and outgoing neighbor requests are also counted towards the limit, so if you suddenly can't add more neighbors, check these sections and clean them up.

To delete a neighbor, click their avatar and choose "Remove from neighbors". Friends from your Facebook friend list can't be deleted from the game unless you unfriend them on Facebook.