Sociology

An Initial Evaluation of Boeings Security and Fire Protection Departments Development Program
Boeing, just like any healthy and progressive organization, has development programs for all its employees. The Security and Fire Prevention Department (SFP) specifically has a development program for their identified high potential employees, which these future leaders must accomplish within 18 months. This development program is meant to develop diverse future leaders within the department.

This development program has been in effect for four years now, and so far there are no published documents evaluating its effectiveness. Looking at the checklist of this development program that each development candidate must meet, the list is long and thus the road towards achieving all these requirements is time intensive. Going through this difficult journey is a necessity to instill the necessary corporate values and skills needed for professional development of our future leaders. The development success of these employees is also dependent on the mentors establishing an effective mentoring program for a more efficient support system for these employees. The goals are to actively engage these employees in company building and support infuse the sense of purpose into these employees and support them individually and collectively towards inspirational leadership.

Nevertheless, due to the lack of published documents, all these suppositions remain speculative, and we do not know if the program is truly effective in its stated goals, or a non value added bureaucracy that ought to be revised. As such, this research is meant to be an exploratory initial qualitative empirical research meant to review the efficacy and effectiveness of SFP Development Program.

Research Question
The main research question to be answered through this study is, Is the SFP Development Program effective in its goal of developing diverse future leaders in the Security and Fire Protection Department In this initial exploratory qualitative empirical research, it would be necessary to ask the following questions

How do managers feel about the effectiveness the Development Program
Do the young employees participating in the program feel about the effectiveness of the Development Program

By effectiveness, we would operationalize it to refer to professional enrichment, cultural enrichment, career development, and career alignment within the company. As such, we can state the operationalized questions in the following manner
Do the young employees feel that the SF Development Program is professionally enriching, i.e., it made them learn more in terms of technical knowledge
 Do the young employees feel that the SF Development Program is culturally enriching, i.e., it made them learn more in terms of company culture
Do the young employees feel that the SF Development Program helped them in putting their careers on track
Do the young employees feel that the SF Development Program helped them in setting their professional goals within the company
Do the managers feel that the SF Development Program contributed a lot to the professional enrichment of the young employees in terms of technical knowledge
Do the managers feel that the SF Development Program contributed a lot to the professional enrichment of the young employees in terms of organizational culture
 Do the managers feel that the SF Development Program clearly puts the young employees careers on track within the company
Do the managers feel that the SF Development Program clearly helps the young professionals set their professional goals within the context of the company

Literature Review
The Need for a Development Program
The field of fire protection has been very active in promoting development programs for fire national protection agencies and academies. A number of studies have been done to look into the need to develop development programs in this realm. Alan Joos (1998) explored the question on whether the Utah Fire and Rescue Academy needs a staff development program in place, what kind of development program may be needed, how such a program may be personally beneficial, and how it may be beneficial to the academy. The author utilized a focused group discussion data collection method which he called the action method, to answer these questions. Regarding the need for a development plan, the researcher found that the lack of a development program has created its own disadvantages such as misinformed or under-informed new employees who may in turn give out faulty information to others. The groups also thought it best to make a distinction between two types of trainings, one for the new staff members, and another one for the current staff members. In each training type, the groups identified the relevant activities and who may be responsible for these activities and classes. Then, the group answered how a development program may personally benefit the employees by stating that such a program may benefit the employees in at least three ways it ensures lessens misconceptions of the new staff it makes evaluation of employees fair by providing additional basis it facilitates team work. Then lastly, the groups addressed the issue of how a development program may benefit the academy that such programs first trains the staff on new technologies it improves customer service the academy gets better chances of retaining its employees and it helps  build a team culture. Overall, this study by Joos affirms that an employee development program is not only a good add-on it is in fact a need to make the hiring and retention of quality firefighters possible. It ensures a certain level of personal satisfaction that makes possible the sense of personal achievement and professional control.

The Need to Evaluate a Developing Program
Deadman (2003) addressed  the problem of a lack of clear succession guidelines, especially for the position of the chief of the fire department. Deadman at the end of the study briefly asks the question, What methods will be used to evaluate the success of the program and emphasizes the importance of evaluation evaluation does not only measure the intended ends of the development program (or in this case, the training program meant to prepare a fire personnel for a system of succession), but also it must measure the elasticity of such a program amidst changes both in the company and changes external to it. Thus, evaluation must begin when the program begins. Its not something done only at the end (Burns-Martin in Deadman, 2003). True to this ideal, Shelton (2001), in her presentation of the historical description of employee development programs, states that its focus has shifted as American economy made its turns. In the 1970s, development programs were in intended to groom young and promising employees for senior management positions (Moses in Shelton, 2001, p. 4).

Thus, a development program is meant to fast-track the training and company experience of these young but brilliant employees, so that the succession does not become a problem and career development meant climbing the corporate ladder. Things changed in the 1980s when hierarchies began to flatten, and thus companies tended to have less bosses. This situation was aggravated by the stock market crash in 1987 when companies needed to downsize (Shelton, 2001, p. 5). During these times, employee development shifted focus from promotion to skill development (Feldman in Shelton, 2001). Thus, in an uncertain world, the value of employee development changed with it. Why companies would spend for employee development that is focused on skills development and not any more on promotion is not a matter we would address, but what is necessary to state here is a validation of what Deadman (2003) stated that an evaluation process ought not only to look at the direct effects of a development program or a succession program it ought to measure the programs flexibility or applicability amidst the changes happening.

From Deadmans statement we also get the insight that evaluation ought not to be done only at the end. Since it ought to be flexible, it is something that has to be in place from the very beginning. Thus, the evaluation process ought to be something that goes along with the development program, checking it for effects and flexibility every now and then. This means that an evaluation ought to be a process, not just a one-shot deal that happens every time one batch finishes the program. It must be more frequent than that, and it must be designed in such a way that it grows or builds up with the development program. This is the very same point that Foxon (1989) made regarding the necessity of an integrated approach or a systems approach in the evaluation of training and development programs.

Kirkpatricks Four Level approach in evaluation is indeed a famous evaluation program which looks into reaction, learning, behavior, and results in the process of evaluating a program (Kirkpatrick  Kirkpatrick, 1998). This approach is retrospective since looking into these four aspects must be done at the end of certain parts of a development of a training program. Evaluation is still clearly distinct from the whole organizational process, and its relation with the development program is not necessarily intrinsic as the results of an evaluation may be considered an end, instead of a means, meaning that it may be used simply as a ritual (Foxon, 1989) before and after the development program. Nevertheless, we still learn something from the Four Levels Approach of Kirkpatrick (1998) we learn to focus on avenues where evaluation ought to be most present.

Deadman presents Pernicks (Pernick in Deadman, 2003) alternative approach of evaluation, or rather a more nuanced approach to evaluation evaluation may be done on five levels, these are the levels of  reaction, knowledge and skills transfer, on-site behavioral change, organizational impact, and monetary return on investment (Pernick in Deadman, 2003, p. 19). As we move from one level to another, rigor and cost increase with the levels. Also, these levels are related to each other such that the lower level acts as a base for the higher level (Pernick in Deadman, 2003). Thus, for Pernick (Pernick in Deadman, 2003), an evaluation process may be done on these different levels, and doing an evaluation on different levels increases in cost and difficulty. Depending on the importance of the evaluation process in a company, i.e., depending on how integrated or systematized an evaluation process is in the development program of a company, and depending on the goals of the development program, the proper evaluation program level ought to be implemented.

Lastly, Shelton (2001) presents the characteristics of a development program. A development program, in contrast with a training program, ought to include learning, career planning, goal setting, and evaluation (Shelton, 2001, p. 11).

Research Procedure
Research Design
The proposed study is an exploratory study that utilizes qualitative research methods to explore the effectiveness of the Boeing Security and Fire Protection Leadership Development Program. This research would specifically explore the following variables professional enrichment, cultural enrichment, career development, and career alignment within the company.

As an initial exploratory research, it works on the first level of Pernicks  (Pernick in Deadman, 2003) levels on the evaluation process, that is, on the reaction level, an exploratory level meant to know how the participants feel or think about something. As such, the data of interest would be the reactions of both the young employees who have just completed the development program and the administrators. Qualitative methods will be used in both data collection and analysis.

This study shall utilize Grounded Theory (Dick, 2005) as the main methodology. As such, the main strategy of the research is to allow themes or codes to emerge from the data gathered which in turn is validated or invalidated by the accumulation of data. It is from these emergent codes that the final paper will be based.

Sampling
Population
The population would be the administrators (managers and members of the SFP Development Group Committee) and the participants and former participants (either withdrew or completed the program) of the SFP Leadership Development Program. All these arehave been directly involved in the Development Program and thus an evaluation of the program necessitates knowing the various experiences of both the administrators as well as the trainees.

Sampling approach, participant selection, and sample size
Purposeful, maximal variation sample (Marshall, 1996, p. 523) shall be employed  both current and past participants, male and female, shall be recruited to make sure that participant reaction is as inclusive as possible. As for the past participants, those who have been promoted after accomplishing the program and those who dropped out shall be recruited. Apart from participants, it is also necessary to gather data from managers and the  development committee members (both of which are called administrators) since these are the other group of people directly involved in the development program. Given constraints in both time and resources, it would be realistic to aim for the participation of three to five administrators (i.e., up to 50 of administrators) and up to ten young employees who completed the program, are currently working to complete it, or withdrew from the program (i.e., up to 17 of the total number of participants).

Data collection
Both in-depth interviews and focused group discussions will be used to collect data for this study. In-depth interviews are most useful in this study since it is a very good tool meant to deeply explore the respondents point of view, feelings, and perspectives (Guion, 2006), which is exactly what we wish to gather here to make a primary evaluation of the SFP Leadership Development Program. The participating administrators, due to the assumption that it would be more difficult to put them together in one group for the purposes of this research, shall be interviewed individually. The young employees on the other hand shall be divided into mini focus group discussions of five. Guide questions were developed for both based on the research questions above. Please refer to appendix 1 for these guide questions.

Procedure
Prospective participants shall be sent invitation letters via email for their participation in this research. The email addresses of the prospective participants shall be acquired through the personnel department, from whom the initial request for employee interviews and group discussions shall be sent. Once emails have been sent and replies have been received, interviews with a duration of approximately one hour shall be set. Focused group discussions each with a duration of an hour or so shall also be set based on the common schedule of the participants. 

The interviews and group discussions shall be recorded via a voice recorder. The permission of the participants shall be secured before recording. This information shall also be stated in the email sent to the prospective participants.

Data analysis
Apart from the data gathered from the interview and focus group discussions, the researchers observations shall also be initially noted, since process notes shall form part of data analysis. The researcher shall use note cards that contains personal notes, as well as codes and other memos, as suggested by Dick (2005). The interviews and group discussions shall be transcribed by the researcher. The transcription shall be kept securely as a password-protected document in the researchers computer. The transcribed data, along with the process notes, shall be interpreted using qualitative template analysis (King, 2004, p. 256), a variant of the open-coding system meant for the Grounded Theory (Dick, 2005).

Validity and Reliability
Validity and reliability are the two central concepts in the assessment of the quality and rigor of a research (Cano, section 5). We shall discuss these two concepts as means to evaluate the strengths and the weaknesses of this research proposal.

Reliability
Reliability refers to the degree of consistency with which instances are assigned to the same category by different observers or by the same observer on different occasions (Cano, section 4). In this particular research, since there is only one researcher, reliability is observed by getting data on both the managers and the young employees utilizing two different data gathering techniques, these are personal interviews and group discussions. The term triangulation has been used to describe this aspect of reliability.  Triangulation has been defined as the collection of information from a diverse range of individuals and settings, using a variety of methods (Cano, section 4). Doing follow up interviews and discussions would have increased the reliability of this research, but the researcher does not have the luxury of time to do the follow ups. Nevertheless, the researcher is confident that using two methods on the two groups of participants makes this research sufficiently triangulated.
  Apart from the use of two data gathering techniques, it is also necessary to address the reliability of the individual research instruments being utilized. By reliability of an instrument, we refer to consistency, the quality of an instrument to produce the same results when employed under the same conditions (Cano, section 4). The reliability of both qualitative interview and group discussion in this research is ensured through the following the use of interview guide and discussion guide and the use of a recorder to record data. These instruments ensure the consistency of data collection in both types of research instruments.

Third, the interpretation of data ought to also be consistent (Cano, section 5). In this research, the use of qualitative template analysis ensures that the data will be interpreted through explicitly set out templates or codes, which should support the consistency and dependability of the process of data analysis.

The consistency of this study that is safeguarded in three different ways as elaborated above, i.e., through triangulation, through the consistency of instruments, and through consistency in the interpretation of data, reassures the reader of the sturdiness of a finding (Lewis  Ritchie, 2003, p. 272) and thus the test of reliability as replication or stability is stronger given that the design of the study is demonstrated to be sufficiently consistent.

Validity
Validity broadly refers to the extent to which an account accurately represents the social phenomena to which it refers (Cano, section 5). When we speak of validity, we may either be speaking of internal validity or external validity. Internal validity is concerned whether you are investigating what you claim to be investigating (Arksey  Knight in Lewis  Ritchie, 2003, p. 273). Simply put, internal validity asks whether ones research does the job it claims to do, whether the research explains or measures what you said you would be measuring or explaining (Cano, section 5). External validity on the other hand refers to the extent to which the abstract constructs or postulates generated, refined or tested are applicable to other groups within the population or to other contexts or settings (Lewis  Ritchie, 2003, p. 273). We shall go through these two types of validity. We shall first discuss the internal validity of this proposal, and thereafter discuss its external validity. 

When we speak of internal validity, two issues arise, and these are the appropriateness of the method to the research question (Cano, section 5) and the validity of the interpretation of data (Cano, section 5).

First, regarding the appropriateness of the method in relation to the goal, qualitative interviews are most appropriate because as the goal of this research is to get the reactions of the participants towards the effectiveness of the SFP Development Program, and as a qualitative interview is meant to provide the descriptions of the life-world of the interviewee, then utilizing this method will at least provide us a peek into the real thoughts and feelings of the participants towards the development program (Kvale in King, 2004, p. 11). As such, it is most important to ensure that the interview would have a low degree of structure imposed by the interviewer a preponderance of open questions and a focus on specific situations and action sequences in the world of the interviewee (Kvale in King, 2004, p.11). The structure along with the questions to be utilized in the proposed interviews has already been elaborated in the research procedures section. Along with qualitative interviews, group discussion (also called group interview Steyaert  Bouwen, 2004, p. 143) shall also be used. This complements the data gathered from the interviews since a group discussion makes the differences and similarities between the different participants, and also the dynamics between the perspectives on a problem, directly visible (Steyaert  Bouwen, 2004, p. 143). If interviews would provide this research the vertical information, the depth of the participants reactions to the effectiveness of the SFP Development Program, group discussions would provide the horizontal aspect of it by providing the similarities and differences of their reactions immediately.

Second, as regards the validity of the interpretation of data, the main issue is tracing the ways by which you have arrived at this particular interpretation (Cano, section 5). Two things ensure the validity of the interpretation of data in this research first is literature review which aided much in illustrating where this research comes from and second is again the use of qualitative template analysis which should make it graphically clear what codes or categories the interpretation takes as relevant.

Next when we speak of external validity, this design of the research being proposed can confidently be categorized to have at least inferential generalizability, that is, the study can be generalized to the wider population beyond the sampling (Lewis  Ritchie, 2003, p. 275). As the research sampling strategically gathers information from the immediate stakeholders and does so via triangulation, we can confidently say that this study is sufficiently inferentially generalizable, and thus, is externally valid. 

Ethical Considerations
There are not many ethical considerations in this research apart from the usual considerations such as ensuring the participants consent in their involvement in either the personal interviews or group discussions the recording of their answers in a digital recorder and the use of their data for the purposes of this research. All these information shall be put in the invitation letter and in their consent forms, both of which clearly states that participation is voluntary and that they could withdraw any time without consequences to them. The privacy of the participants shall be ensured since their data will be anonymized and the transcription of the interviews and discussions shall be kept in a password-protected document in the computer of the researcher.