Regression analysis is probably the most commonly used statistic in the social sciences. Regression is used to evaluate relationships between two or more feature attributes. Identifying and measuring relationships lets you better understand what's going on in a place, predict where something is likely to occur, or begin to examine causes of why things occur where they do.
Ordinary Least Squares (OLS) is the best known of all regression techniques. It is also the proper starting point for all spatial regression analyses. It provides a global model of the variable or process you are trying to understand or predict; it creates a single regression equation to represent that process.
See Regression Analysis Basics
and Interpreting OLS Regression Results
Mitchell, Andy. The ESRI Guide to GIS Analysis, Volume 2.
ESRI Press, 2005.
Wooldridge, J. M. Introductory Econometrics: A Modern Approach.
South-Western, Mason, Ohio, 2003.
Hamilton, Lawrence C. Regression with Graphics.