Make sure that the bike you choose is the right size for you. Normally being able to touch the floor comfortably with the balls of your feet is a good guide. If you also want to ride the bike 'off-road', you may want to consider using a mountain bike.
Check that the tyres and brakes on your bike are in good condition. Bike shops can offer advice and service your bike.
Always use front and rear lights in darkness or poor light.
Buying a second-hand bicycle is a good way to get into cycling without spending too much.