Program Evaluation Final Project

CSC230 Database for Web Applications

EDLD 808 Program Evaluation and Human ResourcesFinal Project

Paul Gruhn, Ed.D. Student – University of Bridgeport, Bridgeport, CT

2

AcknowledgementsTo all the people who helped me ‘do my homework’ – Thank you.  Dr. Linda Paslov, for the class learning experience and final project

assignment with the flexibility to apply this to my context and academic/professional goals.

My EDLD 808 fellow students, it’s been a fun class. Prof. Richard Gnall, for allowing me to use my actual CSC230 class for this

project. Tim Boto, for doing an honest professional level evaluation of the CSC230

course design. Mostly, to all the past CSC230 students who participated in providing honest

feedback and continue to challenge me to be the best teacher I can be.

3

Disclaimer Paul Gruhn, the author of this document, is a doctoral student in Educational Leadership Doctoral program at the University of Bridgeport in Bridgeport, Connecticut. This paper is in partial fulfillment of the EDLD808 Program Evaluation course he is took in the spring 2016 semester. Paul is also an Adjunct Professor at Manchester Community College (MCC), Manchester, Connecticut. The program being evaluated by Paul is a course he has taught for the past three years in his role as adjunct professor at MCC. While the data, research, and evaluation are as accurate as possible; this paper is to be viewed as a student work, and due to limited timeframe should be considered incomplete and stills a work in progress. The University of Bridgeport, or Manchester Community College does not endorse this document.

4

Definitions Blackboard – Refers to an online learning management system (LMS) provided by

www.blackboard.com, this is the LMS, which Manchester Community College currently uses. Creditability – Judgments of quality to determine the level of truth (Patton, 2002). Internal Validity – To show the results achieve were due to the implemented program, and that

other factors did not impact the final outcomes (Posavac, 2015). Mixed-Methods Research – Research which includes both statistical quantitative and thematic

qualitative data, as well as, incorporating both quantitative and qualitative methodologies in gathering and analyzing the data (Teddlie & Tashakkori, 2009).

Outcomes – The realized results/outputs achieved because of the implementation of the program being evaluated (Posavac, 2015).

Program Evaluation – “Is a collection of methods, skills, and sensitivities necessary to determine whether a human service is needed and likely to be used, whether the service is sufficiently intensive to meet the unmet needs identified, and whether the service is offered as planned, and whether the service actually does help people in need at a reasonable cost without unacceptable side effects” (Posavac, 2015)

Qualitative data – The use of words, narrative, and story to measure results in research and evaluation.

Quantitative data – The use of numbers to measure results in research and evaluation. Structured Query Language (SQL) – Is the ANSI (American National Standards Institute)

computer language to communicate with a database, it is the standard for most relational database management systems.

Triangulation – The use of multiple sources, and methodologies to create coherent justification of the presented research themes and results (Creswell, 2014).

5

Evaluation Summary

Manchester Community College

6

Program Description: Academic Setting

Manchester Community College, located in Manchester, Connecticut, was founded in 1963 and serves over 15,000 students a year. MCC offers associate degrees in over 40 disciplines, in both arts and sciences, as well as various certificate programs.

The evaluated program, “CSC230 Database for Web Applications” (CSC230) course, in this document is one of the required three credit courses in the web certificate program offered by MCC.

More information about MCC can be found on their website, http://catalog.mcc.commnet.edu/index.php?catoid=6 .

7

Program Description: Target Population

The target population for both the web certificate and CSC230 course are people desiring to obtain the needed web development skills in order to increase their technical skill set for self and career advancement in their current job, or to enter into a new career in the web technology field.

“Positions for web developers are expected to increase by about 38 percent through 2016 because of the increasing necessity for the internet and internet applications … Web designers typically have a background in computer languages in addition to an understanding of computers, computer programming and computer development … “

8

Program Description: Context

 The online course CSC230 – Database Concepts with Web Applications, is a component of the larger academic offerings provided by Manchester Community College (MCC) listed in the figure below. CSC230 is a required course for the following program offerings at MCC:  • Prerequisite for CST258 Internet Programming course. (Appendix C)• Web Certificate Approval Form (Appendix B), Requirements Form (Appendix D)• Computer Programming Technology A.S. Degree (See Appendix E)• Computer Programming Technology Certificate (See Appendix F)      

9

Evaluation Questions: Student Outcomes

1. Did CSC230 adequately prepare students to take the CST 258 Internet Programming course?

2. Did students earn the MCC offered Web Certificate?

3. Did students earn the MCC offered Computer Programming Technology A.S. degree?

4. Did students earn the MCC offered Computer Programming Technology certificate?

5. Did students skill growth in using SQL related database software?

6. Did students advance in their current job settings?

7. Did students acquire a new job because of the CSC230 course?

10

Evaluation Questions: Course Evaluation

1. Is the online mode of education effective for the CSC230 course?

2. Is the current course design effective in presenting the information that facilitates student learning and success?

3. What are the strengths of the current CSC230 online course?

4. What are the weaknesses of the current CSC230 online course?

5. What are the attitudes and responses to this 3 credit, 15 week, delivered in an accelerated 8-week time frame?

11

Evaluation Design

12

Evaluation Design

Overall Approach

In order to ensure evaluation and overall research credibility, as well as, internal validity, a mixed methods approach was seen as the best methodological choice for this evaluation. As shown in the figure above, triangulation was achieved by the implementation of multiple methodologies, multiple data types, and multiple program participant groups evaluated.

13

Evaluation DesignProgram Records: The first leg of this evaluation was the use of various program records consisting of; student evaluations, student grades, emails from the students, and academic graduation records.

External Expert Evaluation: The second leg of this evaluation was the use of an external subject matter expert. Tim Boto (name used with permission) is the Assistant Director of Educational Technology & Distance Learning at MCC.

Quantitative: Survey to Past Students: The third leg of this evaluation is a quantitative survey sent to 77 past CSC230 students, of which 12 percent did respond. This survey was also used to solicit potential focus group members. Due to limited time and response, a focus group session was not held.

Qualitative: A Case Study, Semi-Structure Interview with Past Student: In lieu of a planned focus group session, a past student provided an insightful, qualitative semi-structured interview provided a case study narrative of her experiences and journey in both CS230 and the web certificate program.

Qualitative: Student responses to a quick email request: A question was asked in an email sent by the professor to students who had already taken CSC230, and were taking CST258 at the time. “Are you graduating, and what are your plans?”

Self-Report: Professor Narrative: As already stated, I am both the evaluator and the professor of the CSC230 course. Self-report narratives can offer threat to internal validity to any study. However, at the same time, a self-report can provide a lot of rich, detailed information, since the professor is the closest to the course details, providing a history across all three semesters being evaluated.

14

Evaluation Findings: Student GradesMeasure Spring 2014 Spring 2015 Spring 2016

Enrolled 24 25 28

Withdrawals 3 (13%) 5 (20%) 4 (14%)

A- or better 18 (75%) 17 (68%) 14 (50%)

B- to B+ 2 (8%) 1 (4%) 7 (25%)

C- to C+ 1(4%) 0 1 (4%)

D- to D+ 0 0 0

F 0 2 (8%) 2 (8%)

Took CST258 TBD TBD TBD

Earned Web Certificate TBD TBD TBD

Earned Associates Degree TBD TBD TBD

15

Evaluation Findings: Student Evaluations

16

Evaluation Findings: What did you like?Students like the learning materials

• The textbook, that the entire book was covered and was easy to follow• Layout of Murach’s book, one page of reading than on the opposite page a shorten outline version.• The online videos, I loved the tutorial videos that the Professor made; I’m a visual learning so the videos helped me

immensely.• The various online tools & supporting technologies• The option to Skype with the professor• Course requirements were stated clearly in the syllabus.

 Students like the Professor

• Professor was timely in providing feedback and returning assignments, was very helpful• Teaching Style • Professor Gruhn was a great professor. He was constantly updating us, making videos, and just making sure he stayed • connected with his students. • Professor’s availability & flexibility (really, thank you)

 Students like the course content

• Working with SQL commands/skills, learning about databases and the back-end• Liked that it was difficult (liked to an extent).• The information learned and how thorough the course was. • The Content (it’s seriously cool to have learned another facet of the Structured Query Language) 

Students like the course structure & design• The technology used in the course supported the goals of the course.• Instructional technology allowed me to achieve my goals.• The interaction • I loved getting the chance to learn by trial and error. The hands on approach really helped and I could work at my own

pace.• The final project was structured but left open ended, which gave me some creativity on what type of database to create• It was straight and to the point. • Structure of the course

17

Evaluation Findings: What did you dislike?Students dislike the learning materials

• Some book confusion• I disliked the database administration portion of Murach’s book. Unlike the previous chapters • we’re everything was written clearly, the last couple of chapters were very confusing.• Uploaded files not being correct• The textbook was sometimes not very helpful.• Sometimes the exercises were not clear enough.• The videos were unhelpful; they were very hard to view and read.

Students like the course structure & design• How to present the work. Screenshots are very tedious• The lack of ability to interact• The discussion boards are hard to interact on without direction; this is not specific to only this class.• Taking screenshots and putting them in word documents (screenshots were fine, but it was a pain to copy/paste them).

 Students dislike the limited amount of time

• The last week of class was crammed and was not a full week.• Too much to do at the end. Was expecting the last week to be a full week.• Class is very fast paced because lot of learning materials got covered in small amount of time.• I disliked the 8 week accelerated time frame for this class. Even though 1st semester HTML/CSS course was extremely hard• I was able to get through with a lot of hard work and not feel rushed. I thought this semester even though the work• was extremely hard I felt super rushed and wished we had more time to take certain chapters a bit more slowly.

18

Evaluation Findings: What would you improve?Learning materials suggestions

• Resolution on videos, it was hard to read the code being written sometimes• The videos were hard to read. Perhaps a different resolution? • To be honest, choose a different textbook (if it’s in your power, that is).

 Course structure & design suggestions

• The syllabus encouraged discussions on the class discussion board, but I find it hard to interact without a reason. • Finding more ways to encourage the discussions would benefit the class participation aspect. • More interaction within groups of students and professor.• At the beginning of the course, make sure the homework sheets align with the book.

 Course length suggestions

• My only suggestion would be to lengthen from 8 weeks to maybe a 10 week course. • That way we can explore this book at a slower pace because there’s just way too much info to be absorbed. • And for those of us with no or very little database experience, the extra 2 weeks can make a world of difference.

• The class can be extended for few more weeks to enhance the learning experience. No suggestions

• I honestly don’t have anything that I truly disliked about this course. Unlike my previous course in Database Design, • this went more in depth. So where in a previous course I had learned about giving, say, Aliases to rows, • I was used to doing it using MS SQL, not MySQL. But seriously, I didn’t dislike anything• This is one of the best books I have had for a class and the videos are awesome. Great job

19

Evaluation Findings: In one sentence• This class was pretty difficult but very rewarding.• If you apply yourself in this course you will truly gain A LOT of knowledge about databases• Tough but fair• I would describe this class as interesting and quite challenging.

• I’ve taken a few web course, and this was little challenging. • This is a class where reading and keep up with the textbook • and lectures really make the most difference.

• This class was fun (most of the time) and very difficult, but enlightening and important.• Very hard working and fast paced.• This class is who is looking to dive into SQL and enhance their own websites,

• or just to learn how to work with databases. • This class was an eye opener to another facet of what my job will be like in the future

• (going to be a DBA, hopefully).• I learned a lot, obviously will have to practice way more to really get it to stick• This class was insanely hard!!!

20

Evaluation Findings: In one word

• Time • Interesting and Quite challenging• Difficult• Busy • Essential (for me to have taken. I really learned a lot!) • Thought-provoking• Unexpected• Interesting• Challenging• Spectacular

21

Evaluation Findings: Outside ExpertTim’s Comments “I am not seeing anything that is wrong, it is good overall.” “Overall this course looks nice.”  (Appendix K1)“You use the ‘Getting Started” section to your own version, not just the canned version presented by the school.” (Appendix K2) “You use announcements a lot” [this is good]

“You use embedded video.” “Your use embedded links”

 (Appendix K3) “Syllabus looks good, and provides the actual links to needed files for the course, and is high up on the syllabus.” (Appendix K4) Course Content

“Has a roadmap in the front”“You have videos in each module”Laid out well, and consistent. (Appendix K5) My Grades Section, he liked how they were organized.

22

Evaluation Findings: Outside ExpertTim’s Suggestions • Switch from Content Folders, to Learning Modules• Consider incorporating additional tools provided by Blackboard

• Tests & Quizzes• Rubrics

• Might fit well with Final Project• Not used a lot by my peers

• Journals

• If this was a full semester course, and not an 8 week accelerated course, suggest using ‘Availability Setting” for modules. When asked, “On a scale of 1 to 5 stars, what would you give this course?” Tim replied, “4 ½ stars, based on my suggestions listed above.”

23

QUAN SurveyQ3 - Did you earn your "Web Certificate" from MCC?

Answer % Count

Yes 31% 5

No 0%  

Still in process 31% 5

Not a goal 38% 6

Total 100% 16

24

QUAN SurveyQ4 - Did you earn your Associates degree from MCC after taking this course?

Answer % Count

Yes 19% 3

No 0%  

Still in process 44% 7

Not a goal 38% 6

Total 100% 16

25

QUAN SurveyQ8 - Did you attend another college or university after taking the CSC230 course?

Answer % CountNo 94% 15Yes - What school and program? 6% 1Total 100% 16

- Central Connecticut State University- 1 Student is also planning on ging to UConn

26

QUAN SurveyQ14 - Did your experience and training in CSC230 help you to get a job in a related career?

I am currently database programmerWorking with MySQL help me by getting familiar working with database, have to use Sybase database at one pointI finished the Certificate Program in Summer 2014 I got a job as web assistant in October 2014Use database/web programming concepts weekly for programming custom Visual Basic application and data mining at work

Answer % Count

No 75% 12

Yes - Describe 25% 4

Total 100% 16

27

QUAN SurveyQ10 - Were you able to use your CSC230 Database training for any personal or other database projects?

Planning on using this to help with a personal business projectWorking on a database at work, but I'm not involved in programming it. At least I understand how the parts all come together.Frequently use SQL and databases in some of my programs.Created a trouble ticket applications that allows me to keep track of clients and computers I do computer repairs forI am a Geek Squad manager and we work with clients who have databases

Answer % CountYes - Describe 38% 6No 63% 10Total 100% 16

28

QUAL Survey

I took this course as part of my Associate's Degree program and hope to use that degree and the information learned from this course and others to break into the software development field.I have a life-long love of learning so there was a positive impact from taking this course, or any other course for that matter.Having just finished the course, I am not sure about the long term impacts. I do see the potential for this course to assist me in my future goals.I enjoyed the classes I rook with you, Professor Gruhn. I earned my degree in Programming, but haven't found a job programming.I ended up going in a completely different direction in regard to career, but I'm glad I have these skills should a new opportunity arise that I wish to pursue.

Gained a better understanding of database creation and organization.

I learned new concepts

Great professor and course learned everything I needed to know about SQL!

Part of my current job deals with an SQL database. I'm sure that taking this course and getting the MCC Certificate helped me get the jobThis course was excellent in teaching programming concepts with databases. It allows me to easily pick up other programming languages and start writing it them faster.The class was designed perfectly to help me understand the basics in creating databases and I found that applying that knowledge to my business has helped with assisting clients more effectively.

29

QUAL Survey

It was a very good class, well above average for online for-credit courses I've taken thus far.

It was a great educational experience and I look forward to the next session.

This was a very practical course. I have a strong understanding of databases.I found out recently that I qualify for the Web Certificate, so I plan on applying for that even though I received my degree in 2014.Enjoyed the course, was challenged, and learned a lot while Prof Gruhn made the course enjoyable.Thanks for the great experience!I thought it was a great course and the Professor and Class made it a good experience. I learned a lot of things that help me in my current job.I think this was an excellent course. I was particularly fond of all the web development courses at MCC, and the professor that taught this course made the learning process not stressful, engaging, and easy to work with the material. The only thing I wish that the web certification program offered was an associate’s degree rather than a certificate. seeing that more and more applications/services are migrating to the, I feel as those this program could be expanded in to a full blown degree programI have had great experiences not only with python and MySQL but overall the experience I had with Professor Gruhn was very fruitful and his way of teaching actually got me to like programming!

30

QUAL Survey: Meet Diane

My goal is to get the web development & design certification here at MCC, and this course is a part of the curriculum. So, I went into this course, not knowing anything at all zero knowledge of MySQL. So, in the fall I took Web Dev 1 and Web Dev 2, and I rolled into your course. I started taking general education courses in 2012 towards a degree, any degree. It didn’t really matter to me; I was doing one course a semester. Then I had a need in my current position I work at UConn. In my department there is a job opportunity within the next probably year, for a publicity and marketing manager. I am currently an executive assistant, and our publicity manager has been mentoring me with the hopes I roll into it. So, when I heard about the certificate at MCC I was really excited because it is helpful in my current position. So, I kind of side tracked from just Gen Ed classes to an actual certification. So, when I am done with this in May I will go back to my courses towards and general studies associates’ degree.

31

QUAL Survey: Email – Will you graduate?

Response: Student A “I am going for a web tech certificate. I may however take a little break, not sure though and probably not but a break feels nice to think about, as I only have the project management after this to do.”

Response: Student B“I'll be graduating this semester with an associate’s degree in Computer Programming Technology (god willing).”

Response: Student C “Thanks for your email.  I have finished all my course work for my Programming Certificate and I am only one course (Project Management) away from the Web Certificate.  I have to talk to advising to make sure I can graduate in August with both certificates.  If I can I will take Project Management this summer. I am undecided about what I am planning in the future.  I want to continue teaching for now, but I may eventually want to pursue a career in web development or programming.  If you have any advice on next steps for either self-study or job paths, I would appreciate it.  I really love programming and have learned so much at MCC.”

32

QUAL Survey: Self-Report!

Enough about me already!

33

QUAL Survey: ConclusionItems to implement before next CSC 230 Course Offering, Spring 2017

• I would like to have one of my Quinnipiac University Online Instructional Designers, who are all trained in “Quality Matters” online course design standards review both my CSC230 & CST258 courses. I wanted to include this in this evaluation. But due to time limitations I could not.

• Update all the CSC230 videos• Use current textbook• Use improved software to improve video quality to High Definition• Add tags to the videos, so students can easily jump to certain sections

• Review and update all homework assignment sheets to insure they are current and accurate.

• Change Blackboard structure from ‘folder based’ to ‘module based.’• Devise a better methodology to communicate to the students, the amount

of effort and time they will need to complete the course. • Develop assignment rubrics to improve the quality and consistency of

grading.

34

QUAL Survey: Self-Report!Open Questions for Continued Discussion

• Some students do not like the accelerated eight-week format, yet completing the Web Certificate program in one year is important to students. One student suggested a 10 week option, which could be considered.

• We need to adhere better to this perquisite requirement. Students from both MCC and Asnuntuck Community College (ACC) who have entered CST258 Internet Programming, without CSC230 have struggled with the CST258 requirements. The problem especially arose with ACC students coming to MCC to meet their CST258 requirement at MCC.

• How can we assure continuity of skill sets across all courses in the web certificate program? As well as, better communication of the various faculty and program leaders? As a teacher in this program, I feel alone.

• Should rolling start dates for the certificate program be considered? This would require offering the courses two times a year instead of one.

• Can we standardize on online course design for all the courses in the web certificate program?

• Students respected the Murach book. We could have a discussion on standardizing textbooks.

35

QUAL Survey: Self-Report!Future Research Plans

• Work with MCC reporting team, to compare course/certificate attendees to actual graduation rates. Did the actuals meet the goals set out in January 2009 (Appendix B)

• Due to time constraints, the two semi-structured interviews were not transcribed. This will be done in the future, and then coded for themes. The findings will then be added to a future revision of this evaluation.

• Expand an evaluation to include the entire MCC Web Certificate program, not just this one course. Did the Web Certificate meet its goals as set forth in 2009? What are the web Certificate program goals for the next 1, 3, 5 years? Is the current program in alignment with current web technologies in play today?

• Include more of the MCC Web Certificate stakeholders in future research and evaluations.

• Can/should the web technologies play a stronger role in the MCC A.S. terminal degree?

• Can I use the MCC Web Certificate program for my thesis? What question(s) need answers?

36

One letter says it all.Hello Professor Gruhn,

Don’t know if you remember me.  I took both CSC 230 and 258 in the spring of 2014.  After that I took Project Management and completed the MCC Certificate Program in the summer of 2014.  I was able to get a job in October 2014 as a web assistant.  It was the first paying job I had since I stopped working in 1998 to be a stay at home mom.  I volunteered in non-profit activities (creating and managing websites) when not working.  Now I work for a NYC based art company in a small satellite office that is about 15 minutes from my home.  I know that completing the MCC Certificate Program helped me get the job.

I completed the survey and would be interested in the focus group if I were available.  Do you have an idea of which Saturday it would be?

I have been at my current job for more than a year and like what I am doing.  I manage an ecommerce website, work with a database (Counterpoint SQL), manage a blog in WordPress, prepare and send out email blasts with mail chimp and do a host of other things.  I got a great evaluation from my boss after just 3 months.  However, the company is not great about compensation so I may be looking at other options where there is more potential for growth.  If you hear of any opportunities or have suggestions, I would welcome them.

Good luck with your Doctorate endeavors.  I really appreciated the CSC230 course and enjoyed the challenges it gave me.  I have no doubt that working on the certificate program helped me get back in the workforce.  It was the first job I applied for (I know I was really lucky!).

Thanks for all you did to make the course interesting and challenging.  Let me know what you decide to do about the focus group.

Best regards,

37

Thank you

Paul GruhnFinal Project EDLD808 – Program EvaluationUniversity of Bridgeport, CT

https://www.linkedin.com/in/paulgruhn