Re: Auditing Quality Systems

From: hmlupin\@libero\.it
Date: Sun Aug 12 2007 - 09:43:02 PDT

  • Next message: Francisco Blaha: "Re: Auditing Quality Systems"

    Dear colleagues,

    This is a very interesting discussion and a very pertinent nowadays. I fully agree with Roy Palmer that the fish industry has been put in a corner by this mushrooming of quality cum safety certification systems. To protract or hide this problem below the carpet, will not be useful at the very end to anybody.

    Moreover, the worst of it, is that there is no assurance that consumers are receiving most out of this additional effort of the fish industry, particularly in terms of further fish safety, since many systems seem aimed simply to create credence in consumers, rather than to achieve some noticeable further risk mitigation, to that already achieved by the fish (and food) safety regulations in force.

    Confusion on the subject seems to me very large, old (since the inception of HACCP regulations) and widespread around the world. Is the common opinion of many Risk Managers (even if it is not easy to find written confirmation), that implementation of the 6th HACCP principle, Verification, to which HACCP Audit and this discussion belongs in first place, is the weakest part of the HACCP chain.

    The subject in turn, touches many different aspects, from internal HACCP audit, structure of regulations and standard systems, to certification and risk. You can start the analysis from different angles, I choose one that seems to me more didactic, even if took me longer than I wanted (I swear I chopped away a lot of text!).

    1. Regulatory safety and quality and contractual (non-regulatory) quality.

    In first place there is the need to separate conceptually between two related, but definitely separated, fields:
    • Regulatory safety and quality (this means attributes in the food required by regulations, either national or from the importing country).
    • Contractual (not regulatory) quality (attributes in the food agreed between the buyer and the seller).

    It can be understood that in industry practice is very difficult, if not impossible, to keep both things separated. It is very difficult for many non-technical managers to understand they are different things, and in most cases (probably the exception are large to very large food companies) it would be un-economic to kept two teams (or two professionals) to deal separately with the two areas. However, both areas follow parallel but different tracks, and technical people in industry should make a mental effort to keep them (at least conceptually) separated and avoid insanity.

    Regulatory safety and quality requirements are codified under the law, and therefore they are at the end, and particularly during critical situations, under the oversight of the sanitary police (call them food and fish inspectors, competent authorities, or wherever you wish, but at the end, even if not necessarily dressed in uniform they are a police, empowered by law to protect public health). For instance HACCP (e.g. in USA, the EC and Canada) in fish and fish products is a regulatory requirement. In a general way they are Risk Management Tools, that government Risk Managers have found more adequate appear as explicit regulations to protect food consumers.

    Contractual (non-regulatory) quality is a different thing. No regulation (in general) is telling you, for instance, how white should be the fillet of the white fish, but theoretically you can define a white colour in terms of XYZ parameters, and a tolerance interval for them, and ask for it. Technically it is possible to sort out fillets by colour; therefore hypothetically you could get them, at a price. Well, perhaps there are not machines to sort out white fish fillets by colour, it will be likely un-economic and it will not matter too much after cooking (however, there are colour charts to check for fillets colour, e.g. very common with salmon and tuna). Provided other factors not be involved (e.g. specie substitution) nobody will get sick because of the different white colour of the fish, but of course you risk the lost of your business, if you do not comply with your contract.

    Contractual (no regulatory) quality attributes is the type of attributes more suitable to be included in Quality management systems. They refer to an agreement between a buyer and a seller, that for instance, could change –in agreement- limits and tolerances, if they so wish, for a specific contract. Actually it is not uncommon today that agreements for 1st, 2nd and 3rd brands (to accommodate food from delicatessen shops to super-discount supermarket chains) could be somewhat different, even if referring to the same purchasing company and the same producer company. There is nothing against public health on that. However, the 1st, 2nd and 3rd brands will have to be, all of them, in compliance with the minimum regulatory public health (and regulatory quality) requirements. There is no compromise on public health requirements. Regulations on food and fish safety apply to all foods regardless their commercial, actual or pretended, quality.

    The first source of confusion is when a given standard incorporates only partially the regulatory requirements, and claim to cover all of them; or because the ones they claim to cover (e.g. absence of veterinary drugs) pretend all the others are covered too.

    The confusion may be compounded in commercial practice by the fact that sometimes the buyers ask (contractually) the sellers for regulatory conditions more demanding than those of the (local or country) regulations. Please notice, it is not the country asking for that, are the buyers (the country is not breaching the WTO SPS Agreement). In turn this is sometimes logical and justified, sometimes it is not.

    For instance most European importers of value-added products ask exporters to comply with EC traceability regulations, even if the main EC regulation on traceability (Regulation EC 178/2002) do not ask specifically for that. In this case this is a logical contractual requirement because it goes in the sense to favour final consumers and make trade more reliable and transparent. In other cases, for instance to ask for zero counts of Listeria monocytogenes in foods that may have it, this means well below the regulatory country limits, for this type of pathogen and product (e.g. smoked salmon), is to play roulette (both the seller and the buyer).

    However, to ask for conditions more demanding than the ones existing in regulations is, something to do very judiciously, and with proper knowledge of existing technology, keeping time limits, economic feasibility, etc. Personally I wonder if this type of extra-safety requirements does not play sometimes, at the end, against the consumer.

    The situation is actually evolving more complex with buyers associations, private food quality associations and supermarket chains developing, selling and or imposing their own quality systems, that may include or not (as own) regulatory requirements. Systems that obviously are not equivalent between them, and that industry is sometimes obliged to adopt them at a cost.

    The validity of all these systems could be seen from the Sudan 1 dye scandal two years ago, that started from UK and covered a number of countries all around the world till reach even China. I think that no “quality” and safety system escaped to be involved, and a number of large food companies in the world were tainted by the scandal too (we talk about sin, not about sinners). The contamination of chilli with Sudan 1 was found by inspection services following routinely services. One of the main faults of the safety/quality systems, as appearing from reports available, was lack of verification of the safety of the raw material, even after the scandal had exploded. The Chinese newspapers made a lot of noise on the subject and put it in the first page. This has been a complete disaster, nobody wants to talk about it, and I can understand that, but people “in the business” can not forget or pretend that situations like that will not happen again.

    There are still many interesting conceptual aspects in the similarities and differences between quality and safety attributes and requirements, but I should focus on those that refer to HACCP Audits.

    2. On the difference between regulations (mandatory by law) and “standards” (contractual, but not mandatory by law)

    The difference and the reason of the difference between regulations (in particular regulations that address matters of public health) have been discussed and illustrated in the previous point. Now it is possible to go a step further and look into the way regulations and standards are written and supported.

    Regulations are not standards, they are not written as standards, they are not necessarily structured as standards and they are normally an interactive part of a regulatory body wider than the support body of any existing quality standard system. What I call here “the regulatory body” is the whole regulatory system, including at least prevention and inspection legs. Many people fail to see “the whole regulatory system”, because they are perhaps too much focused on single sub-systems (like HACCP) or sub-sub-systems (like traceability of live bivalves).

    In turn any existing regulatory system reflects the particular historical development of a given country, in terms of food (& fish) safety; and they are “onion-like” structures, with successive layers, reflecting that development. As food safety in a country is not only the outcome of regulations but also of cultural traditions, and more recently of risk communication campaigns, the situation could be in practice more complex to read.

    Cultural traditions affect things like what, when and how much to eat; this in turns change the potential dose ingested of a hazardous substance or a pathogenic bacteria, which in turns makes a given hazard more or less risky, sometimes enough variation to shift from a Public Enemy to a an acceptable risk. For instance, the difference in the risk of Vibrio parahaemolyticus (Vp) in raw oysters between the USA an Europe is not with the Europeans, the oysters, the strains of Vp, the sea water temperatures or the salinity; it is in the number of oysters consumed by servings.

    What provides the “accepted level of protection” to the consumer is the overall system (included the cultural and risk communication component) and not just the HACCP-based and the Hygiene systems alone, or some other part.

    We can call single regulations “standards”, and actually many do, sometimes as synonyms of regulations, but this terminology is a bit confuse, particularly because “standards” are the key brick of (non-regulatory) quality systems. “Standards” have a meaning only with reference to a specific standardization system.

    In this context, a very important question to answer could be: Is it possible to standardize regulations in particular (e.g. the HACCP-based regulation) and the regulatory system in general (or at least covering all related aspects for a single sub-system)? This means to “translate” a regulation from a “regulatory system” to the “language” of the standardization system X.

    It could be possible standardize isolated regulations, it depends very much on the type of regulation, how the regulation is actually written, self-standing regulations would be easier to convert in standards, but in general it will be a very difficult task. The proof to that is the effort taken to standardize HACCP in the context of ISO 22000.

    It is necessary to point here, that ISO standardization has been with reference to the HACCP guideline system according the Codex Alimentarius, not with reference to actual HACCP-based regulations. The HACCP system as appear in Codex: “Hazard Analysis and Critical Control Point (HACCP) system and guidelines for its application” is not a standard, as clearly explained in the Codex document: “The principles of the HACCP System set the basis for the requirements for the application of HACCP, while the Guidelines for Application provide general guidance for practical application.”

    “Practical applications” became regulatory applications when HACCP-based regulations were approved and put in force. I call here HACCP-based (system) regulations, this means regulations based on HACCP, to differentiate from the HACCP system as it appears in Codex.

    >From a practical point of view, and particularly from the point of view of HACCP Audit, this is very important. It is possible to audit against standards, it is possible to audit against a regulation, but it is not possible to audit against a guideline. It is not possible to audit a guideline because you need at least some “must” and “shall” in the regulation or standard texts. Therefore a first step to arrive to ISO 22000 was to convert the HACCP System according to Codex, in a standard.

    However, not necessarily the “shall” and “must” that you could adopt (e.g. each of the private standardization systems) will correspond to the “shall” and “must” (not to say “should” and “may”, possible alternatives, implicit and explicit branching to other regulations, and very specific requirements – e.g. for imports and/or exports-) of an actual HACCP-based regulation. The “must” and “shall” that come out of a HACCP-based regulation are usually much more than the ones one could be imagined from the HACCP and Hygiene Codex guidelines.

    Definitely I do not want to argue on this argument, personally I think that ISO has arrived to ISO 22000 in all professional seriousness and responsibility, and in certain way has done what could be done adopting a standard system to replace a regulatory system. Their diagnostic in the sense that there is a problem in the industry to handle simultaneously safety and quality is basically correct, at the same time ISO has demonstrated an effort to evolve from the classic ISO 9000 series. I can not say that ISO 22000 will be the final answer, or that perhaps there is further evolution waiting in the future. Nevertheless, I will be definitely reluctant to extend the same type of recognition to many of the systems around, pretending to do the same job (food safety). For instance, some of the systems supporting “organic” foods (supposedly the “safest”) have been already debunked in different countries.

    My observations are on the limitations of the standard systems to address food safety completely, or perhaps to express it better are observations from a meta-system point of view. The meta-system point of view is obviously Risk Analysis.

    3. Audit of HACCP-based systems (in general)

    The discussion about what is, or can be taken by a standard, particularly in relation to the Audit of HACCP-based systems is important because, in order to perform one audit is necessary three basic elements:

    1. A written plan of what are you doing in terms of a HACCP-based and Hygiene (or SSOP) (that in turn shall be a text related to your product and process and to the specific HACCP-based regulation, either of your country, or the country you are exporting your goods).
    2. The HACCP-based regulation of your country and any ancillary and related regulation to it (e.g. regulations about limits of contaminants, maximum bacterial counts, sampling procedures or wherever it could be of relevance to your product/process) to which your HACCP and Hygiene (or SSOP) relate.
    3. A procedure to compare (1) and (2) systematically and professionally, not only in terms of text but in practice (what are you really doing and what authorities are actually asking for). This part can contain provisions on the characteristics and way to train/select a HACCP Auditor.

    The formula is very much the one adopted for Quality Audits procedures (e.g. ISO), but whereas regulations are relatively open and flexible on how to perform internal HACCP Audits (point 3); they are not open and flexible regarding points (1) and (2). Points (2) and (1) are related, but obviously point (2) is the axis, in the sense that for a positive outcome of the HACCP Audit, (1) should mirrors what is requested in (2).

    Moreover, if the audit is in agreement with one HACCP-based regulation, it does not mean necessarily, that it will be in agreement with a different HACCP-based regulation. For instance, it is very well known that for instance FDA HACCP-based regulation for fish and fish products it is not exactly the same that the (now) general EU HACCP-based regulation. They are both HACCP-based regulations, they have almost the same 7 principles, they are referred to food, and they address almost the same hazards in those foods; but, when compared they look more like cousins than homozygote twins.

    Of course, you can invent your own rules and standards (point 2) and ask the industry (point 1) to follow them. You can audit the resulting systems with a procedure similar to (3). Non-regulatory audits, based on a limited number of “standards”, in the other hand, are relatively easy to audit, very often canned check lists are enough to perform the job. In some cases you do not need even a food related university graduate to do the job, or somebody trained in the industry or the food you are dealing with; just somebody trained to put forward questions and tick yes or not in a form.

    But perhaps the most important aspect is that the answers to this type questions and even the outcome of this type of discussion is totally irrelevant to inspection services or regulatory bodies; they will continue to perform inspection and HACCP Audits, following the regulations as they are written. I guess that is the cheapest and safest way to do it, even for industry. Because despite all the systems they could arrive to pay, liability in the case of food outbreak will be always solely with the industry (if not convinced, please read the small letters in your certificates and contracts).

    For regulatory bodies the “certificates” issued regularly by private companies assuring the compliance with HACCP or anything else related to public health are worthless, as official formal certificates. At it best, they may fulfil, for instance, the requirement of internal HACCP Audit (see next point). This means an activity performed by a 3rd party on behalf of the processor. The liability continues to be with the processor, if something is wrong.

    The problem, as already expressed by Roy Palmer, is with the industry that buys, or has to buy (obliged or almost obliged) systems that “assure” food safety (and quality).

    4. Audit of HACCP-Based systems (in particular)

    What to do will depend, in practice, of each particular regulation and on the specific HACCP and Hygiene plans (e.g. the plant HACCP plan could ask for a HACCP Audit frequency higher than the one required in regulations).

    In the case of USA FDA HACCP for fish and fish products (21 CFR Part 23 Sec. 123.8 Verification), the “HACCP Audit” is called “reassessment”:
    “a. Overall verification. Every processor shall verify that the HACCP plan is adequate to control food safety hazards that are reasonably likely to occur, and that the plan is being effectively implemented. Verification shall include, at a minimum:
    (1) Reassessment of the HACCP plan. A reassessment of the adequacy of the HACCP plan whenever any changes occur that could affect the hazard analysis or alter the HACCP plan in any way or at least annually.” (Bold mine)

    It may appear that “reassessment” refers only to a text review. However, from the regulatory text it is clear that effective implementation and processing methods or systems shall be checked too. This means, as it is otherwise obvious, to check actual operations too. Moreover, FDA inspectors, when performing a HACCP Audit, do not read first the HACCP plan you have, they see what you are doing in practice in the plant, make their minds about the kind of HACCP and SSOP plans you shall/should have, and only then put forward questions and look at your written plans.

    I take advantage here, to point that the responsibility for “reassessments” is not with FDA inspectors, but with the processor [read again 123.8, a (1)]. There is a lot of confusion with this too, not only with FDA HACCP regulations, but also with the rest of HACCP regulations all around the world. The fact that fish (or food) inspectors perform, from time to time, regulatory HACCP Audits does not mean that this makes unnecessary internal HACCP by the processor, or that regulatory audits can be taken in lieu of internal HACCP audits, that are part of the HACCP system. Depending on the country, the number of industries, the possible hazards (or risk) associated with the products, the previous records of the company, what is established in regulations, etc. regulatory HACCP Audits can happen once in a year, sometimes once every two years or even longer.

    5. Conclusions

    I am aware that in this type of discussions is nearly impossible (if not impossible) to reach consensus. This could be words against words and people can climb mirrors defending their own business or their point of view. Some quantitative assessments seems to be soon or later necessary.

    The equivalence of the different systems in terms of risk assessment should be the answer. As it is expected it could be done to find the equivalence between different regulatory bodies (working based on Risk Analysis); the same could be applied to private systems that claim be covering safety and hygiene issues.

    At that point in time we will have to reformulate our HACCP Audit (regulatory or internal) approach in terms of risk level too (or more precisely in terms of Food Safety Objectives – FSO-).

    The relation of this aspect (risk) with HACCP Audit can be understood from the following: there are at least three aspects to look for accomplishment of a HACCP-based system in a plant, from outside itself. They are:

    • Suitability/ Adequacy of the HACCP plan to the product/ plant and to the HACCP-based regulation of reference
    • Adherence to the agreed HACCP plan in actual day-to-day operations.
    • Effectiveness of the HACCP plan to contribute to reduce the risk(s) associated with the target hazards in the product (as indicated by the Hazard Analysis of the HACCP plan).

    Today regulations ask (if they ask) to audit for suitability/ adequacy and adherence. Assessment of effectiveness is ahead in time (but coming). The difference between to be suitable/ adequate and also adhere to a given system; and to be effective too; could be the difference in swimming to be just afloat and to prepare for the Olympic gold medal. Therefore, regulatory (and not regulatory) dispositions aimed to food safety shall be scrutinized through the following questions:

    (i) Is this a risk management tool, or part of a risk management tool?
    (ii) How effective it is? And, can effectiveness be assessed in terms of risk?
    (iii) Are there some alternatives for the same purpose? And how we choose the alternatives? Is there some more effective alternative in terms of costs-benefits?

    Effectiveness assessment is crucial to know if industry is investing money in something worth of it, or if only is contributing to enhance credence campaigns of the seller. Of course effectiveness assessment is necessary to separate sharply what is a logic model implementation to mitigate risk (derived from Risk Management) from a marketing construction aimed to create credence in consumers (below the otherwise, always existing, public health regulatory system umbrella).

    The point is that all serious food hazards (those that appear in epidemiology lists) have today already a very low risk. We are talking of (life threatening) risk values – in developed countries- of the order of 1 in 1 million or 1 on 10 million of inhabitants per year, or even less. In average (all foods, all food hazards), in developed countries, food is safe at about 0,999 level (no hospitalization no life threatening), many specific food (e.g. the white fish fillet of the example) are even much more safe (e.g. the hazard could be a pin bone). Zero risk does not exist at all, at the bottom line, even if everything is perfect, you have choke, a physical hazard created by the fact of eating (and there are a number of people that die every year out of choke). Low risk means in practice that to mitigate further the existing risk, it is for instance necessary some true technical breakthrough, be in terms of processing machine, handling and processing, logistic or eventually packaging.

    Regarding non-regulatory quality attributes, private quality systems will have always a role to play, because in that case the standard-based system is very well suited, since the only things to determine are, in the case of audit, suitability-adequacy of the quality plans to the standards chosen and the adherence to them in practice. Once determined that the colour fillet system is all right, e.g. 86 % always within standards, and according to contractual tolerance, that is all.

    Finally, I would like to express something, which is buzzing in the back of my mind since a lot of time ago. I am not completely sure that systems that claim to improve and/ or assure food and fish safety (I do not say here quality) could be sold and imposed always, free and without the oversight of inspection services (or competent or public health authorities). After all, this is at the root of the problem: the possibility to get a safety certification without to pass through public health oversight.

    I apologise because I could not synthesize more briefly my thought. Kind regards to all.

    Hector M. Lupin
    (As Independent Consultant on Food Risk Analysis this time)



    This archive was generated by hypermail 2b29 : Sun Aug 12 2007 - 09:46:26 PDT