Performance experiments can involve multiple execution runs where parameters such as execution platform, measurement tools, methods of measurement, application parameters, and analysis techniques can vary. In order to manage the layers of complexity involved in experimental setup, execution, and post-analysis, a degree of abstraction and automation is necessary at each stage. A layer of abstraction is needed to hide the intricacies involved in experimental set-up and runs for varying sets of experimental parameters. We present an integrated component-based environment that automates the process of running multiple performance experiments and parameter selection of parallel scientific applications. Our toolkit will enable application scientists to easily modify the experimental parameters over multiple execution runs and to selectively retrieve the data for analysis and generation of performance models.