Rounded Corners in Photoshop

(1) Rounded Rectangle Tool (Photoshop 6+)

1. Select the rounded rectangle tool from the toolbar (pictured right).
2. In the options bar, select the desired roundness by adjusting the radius value.
3. Draw a rectangle.
   

This gives you a photoshop shape which is quite flexible. However if you want to make it into a bitmap shape, right-click the layer in the layers palette and select "Rasterize layer".

(2) Use the Selection Tool

1. Select the rectangle marquee tool (pictured right).
2. Draw a rectangle. slightly inside the border of the photo
3. From the main menu, select Select->Modify->Smooth. Enter the desired corner radius. (10 is good)
4. Select the inverse which means from the new square to the outside.
   {Select Menu - Inverse} or ( Ctrl+Shift+ I )
5. Fill the selection with whatever color you want. to create a border or press Delete
   
   Examples: Original
   
   
   Filled Border:
   
   
   Deleted border: