Visual tracking in video sequences is a widely developed topic in computer vision applications. However, the emergence of panoramic vision using catadioptric sensors has created the need for new approaches in order to track an object in this type of images. Indeed the non-linear resolution and the geometric distortions due to the insertion of the mirror, make tracking in catadioptric images a very challenging task. This paper describes particle filter for tracking moving object over time using a catadioptric sensor. In this work different problems due to the specificities of the catadioptric systems such as geometry are considered. The obtained results demonstrate an important improvement of the tracking accuracy with our adapted method and a better robustness to clutter background and light changes.