Loading and unloading container productivity is basically important, because it is support enter and exit of a ship successfully. Cost, time affectivity which is appear during loading and unloading container is give an influence to the efficiency and affectivity in every work. Productivity in a container harbor becomes a work measurement in a container harbor which has correlation in a complexity to take a responsibility in a long period.This observation and analysis started with collected data by responder high value level and low value level in two research places. The observation and analysis bagan with collecting the high level value and low level value data from tow different location, PT. TPS and PT. JICT.The objective of this search are, (1) To estimate the institutional factor regarding the productivity of the loading and unloading container activity; (2) To estimate and analyze the physical factor which consist of the container yard, equipments, gates berth, vessels, and labor regarding the productivity loading and unloading container; (3) To study and analyze the external factors such as economic macro factor and weather factor; (4) study the correlation between the three factors in loading and unloading productivity; (5) In addition to measure differences of worker productivity high and low value level in container both of container proses loading anf unloading terminal is intended.Data which are derived from this search are cross section and time series, which is collected from primary data which allocated a questions sheet to 287 respondents and secondary data which is revelvant and have a correlation with this research.Analysis techniques that applied in this research are Confirmatory Factor Analysis and Full Model from Structural Equation Model (SEM) which is supported by SPSS 14, AMOS 4.01 and MANOVA.There are several result that obtained from this research, (1) Institution have significant effect to container loading and unloading productivity in PT. TPS and PT. JICT; (2) Physical factor which consist of container yard factor, equipments/cranes, gates berth, vessels and labor flow constantly to loading and unloading productivity; (3) The indicator which is derived from external factors mostly does not affect to loading and unloading container productivity; (4) have a strong correlation to container loading and unloading productivity, from the three factors which are shown above; (5) There is no significant difference, between worker on high level stratum as well as on low level in case of handling the productivity of loading anf unloading container activity.