Complex activities are activities that are a combination of many simple ones. Typically, activities of daily living (ADLs) fall in this category. Complex activity recognition is an active area of interest amongst sensing and knowledge mining community today. A majority of investigations along this vein has happened in controlled experimental settings, with multiple wearable and object-interaction sensors. This provides rich observation data for mining. Recently, a new and challenging problem is to investigate recognition accuracy of complex activities in the wild using the smartphone. In this paper, we study the strength of the energy-friendly, cheap, and ubiquitous accelerometer sensor, towards recognizing complex activities in a complete real-life setting. In particular, along the lines of hierarchical feature construction, we investigate multiple higher-order features from the raw sensor stream (x, y, z, t). Further, we propose and evaluate two SVM-based fusion mechanisms (early fusion vs. late fusion) using the higher-order features. Our results show promising performance improvements in recognizing complex activities, w. r.t. prior results in such settings.