Eradicating certification backlog in TVET colleges progress report: DHET, Umalusi & SITA briefing

Meeting report

Briefing by the DHET, SITA and Umalusi on the Certification Status: NC(V), NATED and GETC

The Chairperson began the meeting commenting on the unfairness of the situation students are experiencing when having to enter the workplace without certificates which show their qualifications, after which apologies were heard. The Minister, Dr Blade Nzimande was absent as he had accompanied the President to the investment conference in Johannesburg from 5-7 November. The Deputy Minister Mr Buti Manamela was currently travelling overseas. The Director General (DG) Mr Gwebinkundla Qonde was also attending the investment conference in Johannesburg. Ms Aruna Singh, the acting deputy director-general for technical and vocational education and training was nominated by the DG to lead the Departmental delegation. Mr Zukile Nomvete, chair of the State Information Technology Agency (SITA) was absent due to the meeting coinciding with a long standing engagement and pressing matter which he had to attend to. The Chairperson commented that apologies such as these were very general and were not normally accepted. A more detailed explanation was required. The delegation who would be speaking to the Committee introduced themselves. During these introductions the Chairperson probed about where the chairman of SITA’s board of directors was and said that citing “pressing matters” as a reason for his absence was unacceptable because what was to be discussed during the meeting was of a serious nature.

Ms Singh began the presentation informing the Committee that there was one combined presentation which was prepared through collective efforts by all the stakeholders involved in the certification issue.

The certification backlog constitutes those candidates who have met the minimum certification requirements for various qualifications and were unfortunately not yet issued with certificates within the three months after results have been approved by the quality assurers.

Candidates may achieve minimum certification requirements in their first / one exam sitting or over multiple exam sittings. The bulk of the students who qualify fall within the first examination. An example of a multiple sitting candidate is where a National Accredited Technical Education Diploma (NATED) candidate passes three subjects in the first trimester (April) examinations and completes the fourth subject in the second trimester (August) examination.

To illustrate a typical example, a NATED student might sit at one exam sitting for all four subjects in order to be issued with a certificate, but in that exam sitting only three subjects are passed and one is missing. That one subject would be passed at another exam sitting at a later date. Only after the subsequent exam sitting, assuming the candidate passes, do they become a qualifying candidate for a level certificate.

Certification criteria are dictated by the particular qualification in question:

- National Certificate Vocational [NC(V)] qualification candidates (Level 2 to Level 4) have to pass all seven subjects at the same level. Candidates must pass preceding levels to obtain Level 3 and Level 4 (L3 and L4) certificates.

-  NATED qualification candidates (levels N1 to N6) have to pass a minimum of four subjects at the same level. Candidates must pass all the preceding levels to obtain N5 and N6 certificates.

-  Candidates for General Education and Training Certificates: Adult Basic Education and Training (GETC: ABET) have to pass a minimum of 5 subjects which is a total of 120 credits.

There is a massive exam cycle each year which characterises the TVET sector. The Department runs nine examination cycles in an academic year. There is no assessment body in the world that runs national examinations of this magnitude in the public sector. There are five periods which involve different qualifications. (This is illustrated in slide 4).

In March, there is a NC(V) supplementary exam. In April there is the NATED Engineering exam. In all cases the certification dates were three months afterwards. In the case of the students taking the supplementary NC(V) exam, they should be certificated in July having been involved in the supplementary in March. The NATED Engineering students should receive the certificates in August provided they have qualified. In June, the NATED Business Studies students are examined, because it is a semester and they should receive their certificates in October. In August there is another session of the NATED Engineering and those certificates should come through in December. In November, all of them come together and are expected to be certified in April the following year. This effectively is how the cycle should work. To provide some background on how the examination process works, it begins with registration of the candidates on the Department’s national examination and assessments system. There is also the setting of question papers, packaging and distribution, writing of examinations, monitoring and reporting as well as irregularities in this process which impact on results and certification. Inside each of these processes, multiple operations are taking place. Marking is a major operation as it happens across the country at various marking centres and this has to be coordinated as there are hundreds of thousands of scripts. The November session in particular is a massive session. Moderation processes are undertaken as well as the capturing of marks. All this has to be synthesised within the major exam system in order to produce results which is the next phase. This phase is very scientific, technical and analytical a big part of which is performed by quality assurers. The DHET is delegated those functions for N4 to N6 by the Quality Council for Trades and Occupations (QCTO). The exams unit performs a good bit of the QCTO’s functions for the time being. Going forward this will have to change.

In relation to the NC(V), the overall examination IT system functionality has been improved and meets most operational certification requirements. The Exams Team (DHET, SITA and Umalusi) is continuing with data completeness checks. This relates to certification requirements achieved over multiple examination cycles and processing of certificates; as well as certification requirements on lower levels which were met after achieving the higher level. Data completeness checks are time consuming as analyses are done on a record-by-record basis. Once the underlying alignment matter is identified, computer scripts are written to do database-wide scans where possible.

In terms of NATED, the overall examination IT system functionality has been improved and meets most operational certification requirements. The NATED Business Studies June 2019 examination cycle (batch certification) has been executed with minimal challenges. Similar to the NC(V), the team is also conducting data completeness checks on the NATED programmes. In terms of the GETC:ABET which is in the adult education domain, analyses were done to identify certification processes not completed by provinces. These were completed on behalf of the provinces and data was aligned to enable provinces to continue with the certification process to reduce the number of outstanding certificates. Provinces are now continuing with their certification processes. The national certification warehouse has been introduced to facilitate certification combinations across provinces. The following challenges previously reported have been addressed: Where candidates offered learning areas on both old and new curriculums not all learning areas were combined. In some cases not all candidate records approved for certification were extracted to be submitted to Umalusi.

The reasons for certification backlogs are not desirable, but there are situations which cause backlogs. The current examination Information Technology (IT) system previously did not adequately meet the functional requirements. The system functionality has been improved over the past few years since addressing the backlog. Data integration is at the heart of everything. The Exams Team are currently working on the new system that will become functional and take over what they currently have.

Several system challenges are being addressed.

In the NATED, some candidates are missed by an automated batch certification process. Some full certificates for candidates qualifying in multiple examinations cycles are not being combined. Processing steps of exam cycles are not being executed or being executed out of sequence. The process almost has to be done manually and because of the massiveness of the system, candidates are sometimes missed.

Examinations irregularities (such as the possible leakage of exam questions) need further investigation for them to be finalised as they impact on certification. If these processes are not finalised, candidates cannot be given their results even if they qualify. Ms Singh stated that if late or partial submissions of internal assessment raw marks on the part of technical and vocation education and training (TVET) and community education and training (CET) colleges are not loaded on the system, it will not be possible to produce a result and ultimately a certificate which would indicate that the candidate has met all the requirements. There are certificates which were processed for certification by Umalusi, but are not yet printed due to non-payment of the certification fees by private colleges. A sense of these numbers and the cost implication was provided in the slides.

Ms Singh discussed further reasons for a backlogs in certification. These include systemic issues in the IT system to correctly identify eligible candidates and submit requests for certification to the quality assurer to ensure certification. There are data inconsistencies and changes. Results as approved and released have been changed when certificates were requested from the quality assurer, so the request for certification was rejected. Some candidates have incomplete results after an examination. The present IT system is unable to combine results over multiple examination periods. Outstanding marks of candidates are not recorded automatically, neither are marks that need to be changed from “absent” to a valid mark. Ms Singh pointed out that Umalusi requires evidence before accepting the change. Changes in captured marks that are not correctly applied on the IT system lead to rejection of certification requests by the quality assurer. When there are outstanding certificates for pre-requisite NC(V) levels, the system cannot issue a higher level certificate. Ms Singh said that some candidates could be certified by Umalusi, but the physical certificates are withheld (they are not printed and issued by Umalusi), due to outstanding accounts of the private colleges which candidates attended.

The NC(V) certification backlog which stood at 40 175 in November 2018, had been reduced to 15 985 by 20 October 2019. To date, this number has come down close to 10 000.

The number of NATED outstanding first-issue certificates for the period 2015 to 2018 had been reduced from 20 351 to 2 198 as at 8 October 2019. This represented 0,08% of the 2.8 million candidates who wrote exams.

The certification backlog for the GETC:ABET at 20 October 2019 was 43 737. These certificates were for the period 2010 to 2018. Ms Singh noted that improvements here did not seem to be significant. Ms Singh provided an explanation for these figures and referred to the capacity of the Department. She stated that the team had prioritised the reduction of the NC(V) first sitting outstanding certificates, and thereafter the NC(V) multiple sitting and NATED outstanding certificates. Since the overall NC (V) and NATED certification backlog has been reduced substantially, the team will now concentrate on the larger  backlog for GETC: ABET certificates.

Ms Singh stated that in 2019, for regular candidates who have written and qualified, the Department has managed to keep abreast of issuing certificates within the 3-month period, except where irregularities appear such as a leakage of papers.

The certification backlog for the private exam centres for the NC(V) and the N4/NSC stands at 4 838 candidates. The results were successfully processed by DHET and submitted to Umalusi. These can be printed and issued to candidates after outstanding fees, totalling about R4.2 million, are paid to Umalusi by the 144 private colleges concerned.

A hundred percent certification in real time, and within three months of the exam, is the target that has been set. DHET continues to work closely and collaboratively with the management of TVET Colleges, Provincial education departments, SITA and Umalusi to improve the certification processes and eradicating the outstanding candidate certificates for TVET and CET Colleges.

Managerial, technical and manual processes have been implemented to reduce the number of outstanding certificates. These include:

-           Bi-Weekly management / progress meeting: DG, DHET representatives, SITA and Umalusi.

-           Weekly technical meeting between DHET and SITA officials to remove all possible barriers that are impacting on the release of certificates.

-           Sourcing of outstanding certificates lists from TVET Colleges and manually processing them, by the Department’s examination officials while SITA is enhancing the IT system. The measure was put in place in the last six months to accelerate the reduction of the outstanding certificates.

Here, Ms Singh noted that TVET colleges generally react when candidates to put pressure on them to get the certificates issued, thus the national exams unit has been asking colleges to issue lists of students who would qualify so that those candidates records can be pieced together manually and the certificate can then be issued.

-           Monthly Certification Task Team (CTT) meeting: To tackle candidate certification issues and challenges in processes blocking the issuing of certificates.

The initiative to improve service delivery to students and candidates includes a system that will be implemented where students will be able to track their own certificates through an e-tracking system. The DHET and the SITA have commenced with the development of a solution that will allow both the colleges and candidates to view academic results and the certification status in a secure manner via the e-Government Portal. This system will provide quicker and easier access to information; and result in a reduced number of queries received from colleges and candidates.

Ms Singh discussed a new update for the IT system which is called the Integrated Examinations Information Technology System (IEITS). The DHET has been working with a private service provider for the development of this new examinations IT solution. The new Exam IT system has already been developed. NC(V) L2 and NATED N4-N6 examination processes (i.e. student registrations, capturing of marks, resulting and certification) have been tested with the user community and Umalusi. The focus is now on data migration from the legacy system, development of training manuals as well as finalisation of testing on other levels and qualifications such as the GETC: ABET. The deployment of the new solution is planned to be implemented in a staggered approach per qualification over a four-months’ period to ensure business continuity, commencing in June 2020. It is expected that all qualifications will be executed on the new IT system for the November 2020 exams.

Key challenges and interventions for the Exams Team are listed in slide 18 of the presentation.

The way forward for the DHET is to improve efficiency in the system and develop an effective communication strategy. Steps will include:

-           The automation of information (an e-tracking system whereby students will be able log and track the status of results, academic transcripts and certificates);

-           Advocacy through training on certification requirements in all regions will be undertaken;

-           Building of capacity at a regional level by establishing Certification Committees to collect accurate data on outstanding certificates;

-           Strengthening the usage of social media platforms to address outstanding certificates;

-           Continuing to collect and analyse data from colleges on outstanding certificates, and process these where the certification criteria are met.

Discussion

The Chairperson commented that he had not seen where Umalusi and SITA featured in the presentation. He added that the DHET, Umalusi and SITA are three separate stakeholders who are working together and that he had expected to get an understanding of the role of SITA and Umalusi in the process of issuing certificates. The Chairperson stated that it was frustrating that the presentation seemed to be an enhanced version of what had already been presented to the Committee before.

Ms Seadimo Chaba, a non-executive director on SITA’s board, answered that the entity has, in the past four years, worked to try and remove the blame game and work together as a team. She referred the question of SITA’s role in the issuing of certificates to Mr Vernon John, Head of the department for application maintenance.

Mr John stated that SITA tracks the processes executed and looks for improvement opportunities. It also looks for where controls can be implemented. SITA looks at the software where business processes are combined with business rules and the solution should meet specified requirements. Since SITA is dealing with old systems which are called legacy systems, where some of the coding of the system is written in Afrikaans, it has to work its way through this to weed out the issues. SITA has figured out quite a few issues over the years.

The Chairperson interjected again, saying that the question was not being answered and asked about the role of SITA in the production of certificates.

Mr John stated that SITA’s role is to provide and maintain the software application that the DHET utilises to produce the certificates. SITA supports the Department in the processing of the data on the system to produce the certificates. This means SITA also needs to produce the data sets which must be sent to Umalusi to finalise the results and certificate for each successful candidate.

Prof John Volmink, the Chairperson of Council from Umalusi, stated that as a quality control council, Umalusi is mandated to quality assure the certificates as has been indicated previously. This is the sub-framework and Umalusi is responsible for NC(V), Levels 2,3 and 4 as well as the quality assurance of the NATED programmes N2 and N3. However, Umalusi only issues certificates for N3. The certificates for N1, N2 and N4, N5 and N6 are all issued by the DHET. In terms of Umalusi’s mandate, it is required to implement and develop policies for quality assurance within its sub-framework. Umalusi must also maintain a reliable database of learner achievements which must be submitted to the national learner record database. In fulfilling the mandate of issuing certificates, Umalusi faces a challenge of not having control over the submission of data sets for certification (this is done via DHET and SITA). In terms of completeness therefore Umalusi cannot comment on the number of outstanding certificates. Moreover, in order to comply, the certification data must be scrutinised and validated before certificates can be issued. Prof Volmink stated that he unfortunately needed to report that several rejections take place before certificates are issued, contributing to the backlog. The certificates that are withheld due to private colleges accounts with Umalusi being in arrears, is not regarded as a backlog.

Mr Sibongile Makhoba, client relationship manager at SITA, added to Prof Volmink’s answer stating that Umalusi’s role was to perform oversight functions over the work of DHET and SITA.

The Chairperson commented that he still had not developed a clear idea of what SITA was responsible for.

Mr S Ngcobo (IFP) said that it was surprising that the stakeholders such as SITA and Umalusi were at the meeting to account, which was not helpful. In the previous presentation, the presenter placed the blame squarely on SITA. Mr Ngcobo said he was surprised that he did not see the Minister, Deputy Minister and DG. He said he took exception to their absence as this meeting was quite important and the invitation for the meeting was extended some time ago. Mr Ngcobo stated that the presenter of the current meeting knew that she was not being helpful as the issue at hand was the backlog of certification and examinations which is what the Committee wanted answers to. The presenter came here to tell the Committee what had been done, how it had been done and how the backlog has been reduced. He added that the DHET has not responded to the vital question of why learners are not being issued with certificates. Mr Ngcobo came across as being frustrated with the presentation. He apologised for responding to the presentation in this way. He questioned whether the head of the DHET was qualified for this position. He added that the head of the Department needed to know why learners were not receiving certificates and explain why he was okay with this. TVET colleges fall under the DHET which is led by Minister Blade Nzimande. Mr Ngcobo said he respected the Minister as he is the person who initiated this approach to vocational education. He had spoken passionately about this on radio and TV trying to convince people that there was nothing wrong with TVET colleges. He was educating people. Now learners get into TVET colleges, but they don’t receive certification. Should the Committee take 50 000 as something which is not sufficiently serious? Should the Committee listen to somebody saying at least they are doing something about it, they are reducing the backlog? Should the Committee listen to someone saying they are all working together as a team? He said a reason for the backlog in certification must be given as the presenter clearly understood what the Committee wanted to know.

Ms J Mananiso (ANC) said she was frustrated about the certification backlog which has occurred since 2007 and asked if that had been attended to. Certification is a right for access to better prospects and she asked why all the institutions were not being given the same attention. She asked how much money has been spent on addressing this backlog of certification. Ms Mananiso commented on an administrative issue with customer service at TVET colleges. She asked if there was any capacity building taking place as the service was not satisfactory. This has caused young people to view TVET colleges as their least preferred option. She stated that the backlog was a reflection of an intensive care unit (ICU) as nothing was changing in terms of the numbers as people were not being serviced as they are supposed to be. She asked about non-compliance as this was not presented on. Ms Mananiso pointed out that e-tracking would not assist those who did not have internet access. She suggested that fieldworkers be sent to register people who are experiencing this issue.

Ms Mananiso added that Mr Ngcobo was correct in venting on the certification backlog and asked for specific timeframes on addressing this issue so that MP’s could properly perform their oversight role.

Dr W Boshoff (FF+) stated that he felt providing an explanation should not be confused with excusing. The multitude of exam sittings has created an issue for data integration which is actually an IT issue. Dr Boshoff asked if there was a way to reduce this issue? What preceded the present system? Because it seemed as though there was a system which worked prior to this. Perhaps it had serious shortcomings and therefore had to be replaced with something better which also seems to have technical issues that are being attended to now. Dr Boshoff asked what preceded the current system and probed whether the current system was better.

Ms Singh answered that there was never another IT system prior to the current one. It was a very old, manual system that was never rebuilt at any stage.

Ms D Sibiya (ANC) asked if there was consequence management which Umalusi implemented against any colleges for not issuing certificates and wanted to know which colleges were involved in this. She asked when the DHET, SITA and Umalusi would eradicate certification issues dating back to 2002 and 2007.

Mr Makhoba said TVET colleges were not the culprit in terms of payment for certificates as provision was already made for this by the fiscus. Defaulters in terms of non-payment are from the private sector colleges. A list can be provided of who is involved as well as the amounts owed.

Mr W Letsie (ANC) stated that it must be noted that the apology of the chairman of SITA’s board of directors was rejected, because the invitation for the meeting was sent a long time ago. If there was a pre-planned activity, the Committee should have been informed in due time and not on the day the Committee meeting was set to take place. Mr Letsie agreed with Mr Ngcobo’s comments and referred to various dates since 2016 on which the DHET, SITA and Umalusi had presented to the Committee on the certification backlog. This amounted to a total of 11 times. These entities had set the date for eradicating the certification backlog at the end of January 2016. He pointed out how the target was never met throughout 2016 and yet today there is still a backlog, despite the IT issue having been identified as a key problem in 2015. Mr Letsie asked for new time frames and if there was a possibility to visit SITA and observe their entire live system.

Ms Chaba made an apology for the presentation lacking an explanation for the backlog and said a broader explanation would be submitted to the Committee. She said that a visit from the Committee would be appreciated and emphasised that SITA was not taking the backlog issue lightly.

Ms N Mkhatshwa (ANC) said she wanted to emphasise what the Chairperson said. Separate presentations from the various stakeholders at the meeting were necessary. She asked how graduates were being worked with in the midst of the crises of the outdated system? She said that perhaps young people could come up with more innovative proposals on how to solve these issues. Ms Mkhatshwa asked about the challenges experienced with the e-portal. She also asked about the data integration issue and wanted to know what interventions were taking place in this regard. She asked about how the change in number of the certification backlog was ascertained.

Ms Singh said that it was noted the Committee would like separate presentations in future.

The e-tracking system is still being prepared and has not yet been launched. It will serve as an additional resource to students. However, the college would remain the first point of contact for students’ queries. There is a very active call centre.

Mr P Keetse (EFF) commented on the absence of leadership when critical issues were being discussed. He said that the Minister, Deputy Minister and DG needed to take the Committee seriously as they were constantly receiving apologies for people being overseas when they were not elected to do this.

Mr Keetse said he found some of the reasons for the delay in certification to be very awkward. He said it seemed like some issues were being hidden. He asked what is so difficult about producing a certificate and referred to the incapacity of the IT department saying that the emergence of the fourth industrial revolution was waffled about every day, but that we are failing to compose a system to ensure that certificates are issued timeously.

Mr Keetse said he found it to be extremely shocking and discouraging that a student would have to beg to receive their qualification after completing their studies, only to be told incorrectly that their marks did not correlate or that they had not sat for a particular exam. He said this issue could be linked back to capacity and he disagreed that there was sufficient capacity.

Ms Singh said that in terms of capacity the Department was grappling with historical neglect of the system. The system was also subject to approval from Parliament which has taken time.

Mr Makhoba agreed that it was painful not to receive a certificate after a candidate had passed all their examinations. He said that the entities concerned could do better as a collective. Mr Makhoba said that data could not be fabricated for certification purposes. It needed to be authentic, as the currency of the qualification must be respected.

Mr B Nodada (DA) said the way the presentation was done was quite sad as he was notified of at least one issue of certification backlog in his emails every day. He asked who the IT person or team was, what their qualifications are and what exactly they were doing. He said that the presentation seemed to celebrate the reduction in the numbers of certification backlog and noted that the Committee cannot celebrate something that should not have transpired in the first place. He also emphasised Ms Mkhatshwa’s comment that everyone did not have internet access.

Mr Nodada said that when the DHET, Umalusi and SITA made a presentation again they needed to indicate what challenges they were experiencing and how they wanted the Committee to assist. He added that even if bold and unpopular decisions needed to be taken, the Committee would do that. Mr Nodada probed about when the entities last had a bi-weekly meeting to solve the issues being experienced. He also asked about the timelines for clearing the backlog so that if this issue was not resolved by the time the next presentation is made to the Committee, there should be consequences.

Ms Singh said that the entities were asking the Committee to keep abreast of whether departments in general and this Department in particular is in a position to upgrade its systems periodically. Ms Singh said a new IT system had to be built. There was no way the existing system could have been modified to deal with the current challenges. The Department is trying to deal with the backlogs at present as it does not have to wait for the new system to be in effect to deal with this.

Mr Ngcobo commented that statistics were not being dealt with, but rather human beings, some of whom were already dead.

The Chairperson commented that commitment must be made to a specific date for when the backlog on certification would be cleared. The absence of the Chairperson and CEO remains a concern for the role SITA is playing as SITA has constantly been blamed for the issue at hand. He also asked about the value of the contracts for the new system.

Ms Singh stated that the Department was hopeful the backlog would end by mid-June 2020 based on their planned activities. She said provision must be made for malfunctions and challenges in the live testing.

Ms Violet Shetlo, from the DHET, said that the value of the contract is R52 million. Three years of the contract was set out for development and it was set to be six years before the system was to go live. The other three years was set out for support. The system is now just being evaluated.

Ms Chaba said that a written explanation would be sent regarding the absence of the SITA chairman within the next week.

Prof Volmink stated that linearity is now taking place – one step leads to the next - and having more exam sessions has had a positive impact. He said the analytics around eradicating the backlog issue was time consuming and the legacy issues were highly specialised. The Department has sourced a service provider to assist with this.

Mr Andre Taylor, senior manager of application maintenance at SITA, said that a backlog began when certificates had to be reissued via Umalusi which is where quality issues came up. The new system also needed to be configured in order to make use of standardisation methodology. Results were also not always aligned according to Umalusi’s calculations, causing an issue with the system.

Ms Mkhatshwa added to her earlier comments on the ongoing referral to an old system and asked why this system could not be changed.