Motion detection in the video is one of the development of pattern recognition science. The motion detection processing to detect moving objects in video. Motion detection of one object on the video avi video is focused to detecting the operation area of moving objects in video avi with the concept separate moving objects with background. Operations used in the motion detection process is operations in image processing.Stages of motion detection process begins with reading avi video, convert video to grayscale image in every frame, from grayscale image each frame is convert to binary image, then look for black and white areas that differentiate between object with the background in each frame, the data area take is the pixel coordinate data for the initial and final pixels of each row and column pixel matrix, the data obtained is used to determine the height and width of the object in each frame. Restricted area in each frame is made by placing the rectangle on the area based on the coordinate matrix and the height and width of the object.