Comparison of Conventional Methods and Automated Systems for Determining Antibiotic Susceptibility of Bacteria Isolated from Urine Culture

Objective: The aim of our study was to compare the results obtained from different methods, which are used to determine the susceptibility to antimicrobials, and to determine the agreement between them. Materials and Methods: A total of 397 strains consisting of 329 gram negative and 68 gram positive bacteria that were isolated from urine samples and identification and antibiogram of which were performed by automated systems, were included in the study. The results obtained from broth microdilution, E test, VITEK and disk diffusion were compared. Results: Our antimicrobial susceptibility test results in gram negative bacteria showed a categorical agreement of 93.3% with a very major error rate of 1.5 in disk diffusion test, while a categorical agreement of 90% with a very major error rate of 2.2% in Vitek 2 test. The agreement for E test was 95.3% with a very major error rate of 1.1. In the antimicrobial susceptibility test of gram positive bacteria, the rate of agreement for disk diffusion, E test and Vitek 2 was 97.2%, 98.4% and 95.4% respectively. Very major error rate was found % 1.5 ve % 2.6 in disk diffusion and Vitek 2, respectively, while no very major error was found in E test. Conclusions: In conclusion, our data suggest that disk diffusion test, Vitek 2 system and E test test are accurate and acceptable for antibiotic susceptibility tests of gram positive bacteria. The agreement rate of BMD and E test methods was found above 90% for all antibiotics, and it was reached a conclusion that the susceptibility results of gram negative bacteria for ceftriaxone and nitrofurantoin obtained from VITEK; the susceptibility results of disk diffusion test for ceftriaxone and ciprofloxacin should be confirmed by a more reliable method.


Introduction
Urinary tract infections are one of the most common among community and hospital-acquired infections. Due to the resistance to the antibiotics used in the treatment of these infections, treatment failures occur 1,2 . Regular surveillance studies will enable the identification of ideal option in the treatment and prevention of treatment failures. In order to perform a successful surveillance, the antibiogram data should be obtained with a sensitive, specific and reliable method.
In routine microbiology laboratories, sensitivity tests with different properties are used depending on the status of the laboratory. Systems with advantages such as repeatability of results, objective results and ability of quick reporting, access to data history and low cost are highly preferable. The disk diffusion method, which is preferred because of being repeatable, easy to apply and low-cost, has some disadvantages such that it is not recommended in some antibiotics and the results are subjective due to the fact that it cannot provide the MIC values and produces inconsistent results. On the other hand, gradient tests, also known as E-tests, have the advantages such as ability to determine the MIC values in addition to the repeatability property which is easy to apply and evaluate. The most important disadvantage is that it is not costeffective. The microdilution method, considered as the gold standard method in antibiotic susceptibility tests 3,4 , is laborious to perform, time consuming and relatively high in cost. In most of the clinical microbiology laboratories in our country, especially where sample flow is high, commercial automated systems are preferred and used because of the advantages they provide to the process. However, there are reports that commercial systems may produce erroneous results for some antimicrobial agents while testing gram positive and gram negative bacteria [5][6][7][8] . There are even some recommendations presented by international standards with regard to that laboratories must confirm certain susceptibility phenotypes by a manual method 3 . The aim of our study was to compare the results obtained from disk diffusion, E test and automated systems by using the gold standard, broth microdilution method (BMD), to determine the susceptibility to antimicrobials frequently used in the treatment of gram negative and gram positive microorganisms isolated from urine cultures. The antibiograms of the strains were carried out by using an automated system (Vitek 2, bioMérieux). The susceptibility results obtained from the automated system were recorded. The susceptibility tests of ampicillin, cefuroxime, ceftriaxone, ertapenem, ciprofloxacin, gentamicin, nitrofurantoin were studied for gram negative bacteria and the susceptibility tests of ampicillin, ciprofloxacin, vancomycin, nitrofurantoin antibiotics for gram positive bacteria using the disk diffusion, gradient diffusion (E test) and broth microdilution methods. a) Disk diffusion method: A day in advance, pure cultures of all of the bacteria to be studied were incubated overnight on the blood agar. The bacterial suspensions prepared in 0.5 Mcfarland turbidity standard were inoculated on the Mueller-Hinton agar (Oxoid) surface. Antibiotic disks (Oxoid) of ampicillin (10 μg), cefuroxime (30 μg), ceftriaxone (30 μg), ertapenem (10 μg), ciprofloxacin (5 μg), gentamicin (10 μg), nitrofurantoin (100 μg) were used for gram negative bacteria and ampicillin (2 μg), ciprofloxacin (5 μg), vancomycin (5 μg), nitrofurantoin (100 μg) for gram negative bacteria. After the plates were incubated at 35ºC for 18 hours (24 hours for vancomycin), the inhibition zone was determined according to the standards 9 . b) Gradient diffusion (E test) test: The bacteria to be tested were brought to a 0.5 McFarland turbidity and spread on the Mueller Hinton agar surface with a sterile swab. Test strips (bioMérieux AB BIODISK) were placed on the agar surface. After the plates were incubated at 35 °C for 18-24 hours, the MIC value was determined. c) Broth microdilution: The stock solutions of antibiotics to be tested (Sigma Aldrich, USA) were prepared in special solvents. The strains were tested in U-based 96-well plates, cation-added Mueller-Hinton broth (MHB) at antibiotic concentrations of 64-0.125 μg/ml. Nitrofurantoin was studied in the range of 256-0.5 μg/ml. A sterility control well and a reproduction control well were left on each plate. The bacterial suspension, turbidity of which was adjusted to the 0.5 McFarland standard, was inoculated in an amount of 100 μl in all wells except for sterility well, making 1:10 dilutions with MHB. The plates were incubated at 35 °C for 24 hours. At the end of incubation period, the lowest antibiotic concentration, in which no visible growth was observed and the growth was inhibited, was recorded as the minimum inhibitory concentration (MIC) value. Evaluation of the Results: Tests were repeated when inconsistent results were found. The susceptibility results in the same category were evaluated as agreement; the results that the reference test found as resistant and other tests found as susceptible were evaluated as very major error (VME), the results that the reference test found as susceptible and other tests found as resistant were evaluated as major error (ME), a strain found to be dose-related resistant with a method and susceptible with another (I-R, I-S, S-I, R-I) was evaluated as minor error (Minor) 4 .

Statistical analysis
Statistical analysis was done by using IBM Spss Statistics for Windows, Version 22.0 Amork,NY IBM Corp. Kappa coefficient was calculated for the statistical analysis of the tests. According to the Kappa coefficient, the agreements were interpreted as follows: 0 ≤ κ < 0.20 No agreement, 0.20 ≤ κ < 0.40 Poor agreement, 0.40 ≤ κ < 0.60 Moderate agreement, 0.60 ≤ κ < 0.80 Good agreement, 0.80 ≤ κ < 1.00 Perfect agreement 10 . Ethics Committee Approval: Ethics committee approval was received for this study from the ethics committee of Istanbul University Göztepe Training and Research Hospital Ethic Committee (03 March 2017, 2017/0119).

Results
The categorical agreement and error rates of gram negative bacteria included in the study are given in Table 1. The agreement range of disk diffusion method with the reference method was between 88.9-98.3%. The highest agreement was observed in nitrofurantoin, while the lowest agreement was found in ciprofloxacin (98.3% and 88.9%, respectively); they were statistically evaluated to be in good agreement and perfect agreement category (0.74 and 0.83) (p <0.01). Considering the agreement of E test method, the highest agreement was observed in nitrofurantoin with 98.6% and the lowest agreement in ceftriaxone with 92.1%. According to the Kappa coefficient, both were found to have a perfect agreement rate (0.89 and 0.83 respectively). When the agreement of the automated system results with BMD method is examined, the lowest agreement was observed in ceftriaxone with 85.7% and the highest agreement in ertapenem with 94.7%. In contrast to these results obtained from Kappa analysis, ertapenem was included in the moderate agreement category (0.43) and ceftriaxon in good agreement category (0.70). The results of nitrofurantoin in gram negative bacteria showed no very major error. The results of ampicillin, ceftriaxone and ciprofloxacin obtained from Vitek 2 showed a very major error rate of 3.3%, 3.3% and 3.1%, respectively. In the disk diffusion method, the very major error rate for ampicillin was 3.3%, whereas the very major error rate was 4% in the results of cefuroxime obtained by E test. Such a high rate may be due to the low number of strains studied . On the basis of antibiotics, the ceftriaxone results obtained from disk diffusion and Vitek 2 (11% and 14.3%, respectively), the ciprofloxacin results of disk diffusion method (11.1%), and the nitrofurantoin results of Vitek 2 were found to have overall error rates of over 10%. Table 2 shows the categorical agreement and error rates of gram positive bacteria. The agreement of disk diffusion method for the antibiotics studied was over 95%; the highest agreement was found in ciprofloxacin with 98.5% while the lowest agreement in vancomycin with 95.1%. The lowest agreement rate of Vitek 2 was determined in ciprofloxacin as 93.9%, while the lowest agreement rate of E test was found in nitrofurantoin as 97%. It was also statistically found that the nitrofurantoin results between disk diffusion test and BMD, and the vancomycin results of E test were evaluated to be in "good agreement" category and the others in "perfect agreement" category. The statistical analysis could not be performed when the frequencies of 0 (zero) were present in the analysis tables (NA). No very major error and no major error was found in any of the ampicillin results in grampositive bacteria. With all methods, a minor error was detected in one strain. Among the methods with which ciprofloxacin was studied, a very major error rate of 1.5% was found in disk diffusion, while no major error or minor error was found. In the E test, no error was found for ciprofloxacin, while Vitek 2 results showed a very major error of 3% and a major error of 3% and no minor error. The highest overall error rate was observed in Vitek 2-ciprofloxacin results (6.0%).Nitrofurantoin disk diffusion results showed a very major and major error rate of 3% totally (1.5% and 1.5%). The error rates for vancomycin were as follows: a very major error rate of 2.4% and a major error rate of 2.4% in disk diffusion, and a major error rate of 1.5% in E test. The Vitek 2 vancomycin results showed a very major error in two strains (3.1%) and a major error in one strain (1.5%), while no minor error was found in the Vitek 2-vancomycin results. When the agreement rates of gram positive bacteria and gram positive bacteria were evaluated totally, it was noted that all three test methods were relatively low in gram negatives. The agreement rate of the susceptibility results in gram positives obtained from disk diffusion, E test and Vitek 2 methods were 97.2%, 98.4% and 95.4%, respectively, while these rates in gram negatives were found to be 93.3%, 95.3% and 90.5%, respectively. Vitek 2 exhibited a very major error rate of 2.6% in gram positive bacteria and 2.2% in gram negatives, and disk diffusion showed a very major error rate of 1.4% and 1.5% in gram positive and negative bacteria, respectively. A very major error rate of 1.1% was obtained from E test in gram negatives, while it was not found in gram positives. Although the very major error rate of gram positive bacteria in Vitek 2 results appears to be relatively higher than that of gram-negatives, it was found that the very major error rate of gram negative bacteria was significantly higher (9.4%) when overall error rates are considered (Table 3).

Discussion
Urinary tract infections are among the most common infections both in the community and hospital. Although the treatment in community-acquired patients is usually planned empirically, especially data obtained from antibiotic susceptibility tests of hospitalized patients' infections are particularly important in the management of infection. The success of treatment depends on the results of antibiotic susceptibility tests provided by clinical microbiology laboratories. Therefore, it is mandatory to surveil the accuracy of commonly used susceptibility testing methods. Automated systems are broadly used in laboratories where workload is heavy. They both reduce workload and provide results in a short time. In addition, the properties such as storing the results electronically and objective reporting are advantageous for users. Compared with automated systems, tests such as disk diffusion test that can be easily obtained by any laboratory and gradient tests that provides MIC results to their users even though they are not cost effective are substantially used in routine laboratories. To form an opinion about the reliability of all these tests, their results should be compared with the results of reference method. In our study, the susceptibility results of 397 urinary isolates, of which 68 were gram positive and 329 were gram negative, were evaluated by using four different methods. The comparison of the obtained results was made by using BMD as the reference method in accordance with the recommendations of the FDA. A study that totally evaluated the antibiotic susceptibility results of different number of strains of Enterobactericae family obtained from automated systems 11 found a categorical agreement rate of 95.7%, a very major error rate of 0.5% and a major error rate of 0.3% and a minor error rate of 4%. In the same study, it was indicated that the categorical agreement rates of beta-lactam group antibiotics were between 98.7-100%, quinolones was 96.2%, gentamicin was 98% and nitrofrontoin was 84.6%. In another study conducted [12], it was reported that categorical agreement rate was 92.9%, and the rates of very major error, major error and minor were 0.3%, 1.6% and 5.1%, respectively, when the Vitek 2 susceptibility results of 138 nonfermenter bacteria and the results obtained from BMD method were compared. In our study, 30 P. aeruginosa strain was studied as nonfermenter bacteria. According to the data we obtained, our agreement rate was 92.3%, no major error was detected, and the rates of very major error and minor error were found to be 1.9% and 5.8%, respectively. Although our agreement rate and minor error rate are similar, our high rate of very major error may be attributed to our small number of strains and antibiotics. In their study on 72 Enterobacteriaceae strains, Lee et al. 13 compared the ertapenem susceptibility results that they obtained from two automated systems (Vitek and MicroScan), E test and disk diffusion methods with BMD, and suggested that the best agreement was obtained from E test, routine tests such as an automated system or disk diffusion had limitations in determining the ertapenem resistance in Enterobacteriaceae and the results should be confirmed by an additional MIC test for an accurate result. In a study conducted in the United States 7 , the researchers reported that the very major error rates that they obtained from Vitek 2 system for cefepim and meropenem were high with 67% and 27%, respectively, and that they found lower very major error rates for cefepim (6%) and meropenem (%0) according to the results they obtained with E test. While they stated that they did not find any major error with these methods, they emphasized that the Vitek 2 results of KPC-producing strains they included in the study were not reliable for the antibiotics they tested, and that the agreement rate of E test and BMD was good. When we look at the overall results obtained from the strains in our study, including Klebsiella strains, it can be seen that our Vitek 2 results in cephalosporins and ertapenem have a lower agreement rate than E test, and that our very major error rate is relatively high ( Table 2). When we statistically interpret the data we obtained from our study, ertapenem was included in moderate agreement category, ceftriaxon was included in good agreement category and cefuroxime was included in perfect agreement category according to Kappa analysis. Lat et al. 7 studied a small number of resistant strains in their study. As we have emphasized, they also evaluated E test to be more reliable than automated system. Stone et al. 14 compared the susceptibility results obtained from disk diffusion and VITEK 2 automated system with the BMD results for 61 isolates of rare Enterobacteriaceae species; they found a categorical agreement rate of over 95% between BMD and disk diffusion, and an agreement rate between 68.2% and 100% in the BMD and VITEK 2 results. In another study 15 compared the antibiotic susceptibility results of 20 Enterobacter cloacae strains isolated from cooked foods with M.I.C Evaluator and BMD method, researchers found the agreement rate of E test between 15% and 100% despite the difference compared to the antibiotics they evaluated in the study. In their study, they found the agreement rate of ciprofloxacin as 90% and indicated that M.I.C Evaluator method had a high efficiency. In our study, the agreement rate of ciprofloxacin susceptibility results of 321 gram negative strains obtained by E test was determined as 94.7%, and it was statistically interpreted to be in perfect agreement category. There are different publications reporting the comparative results of the susceptibility test methods, also used for gram positive bacteria. Tenover et al. 16 found the agreement rates of Vitek 2, E test and disk diffusion tests as 93%, 90% and 88%, respectively, in their study in which they investigated the agreement of six methods for linezolid susceptibility of 50 staphylococci strains and 50 enterococci strains. In a study compared the susceptibility results of 1248 enterococci strains from our country 17 that were obtained from automated system and BMD, the agreement rates for ampicillin and vancomycin were found to be 97.6% and 99.2%, respectively. The very major error rate in ampicillin was 1.5%, while it was not detected in vancomycin; the major error rate was found as 0.6% in both antibiotics, and two (0.2%) minor errors were detected only in vancomycin. In our study, vancomycin was tested for a total of 41 enterococci strains (32 Enterococcus faecalis and 9 Enterococcus faecium) and 27 Staphylococcus aureus strains, and ampicillin was tested only for enterococci. The agreement rates obtained from vancomycin and ampicillin were 95.4% and 97.4%, respectively. A minor error rate of 2.6% was found in the ampicillin Vitek 2 results, no minor error but a very major error rate of 3.1% and a major error rate of 1.5% were found in vancomycin Vitek 2 results. Although our data were in parallel with the FDA recommendations, it was reached a conclusion that our small number of strains may have caused a relative elevation in the very major error rate. Kirn et al. 18 evaluated five methods including E test for the daptomycin susceptibility results of a total of 149 Enterococcus faecalis, Enterococcus faecium and Staphylococcus aureus isolates, and reported essential agreement rates of 63%, 83% and 100% between BMD and E test for Enterococcus faecalis, Enterococcus faeciumve Staphylococcus aureus, respectively. They emphasized that the nonsusceptible results determined against the tested drug especially in Enterococcus faecium strains should be confirmed by the reference method for the other methods they tested in their study along with E test, and that E test would increase the agreement a little in cases where the reference method cannot be obtained. In another study 19 , the test results of 134 Staphylococcus spp. and 84 Enterococcus spp. strains were compared with the Vitek results; 9 very major errors (2 inducible clindamycin resistant strains), 4 major errors and 30 minor errors were found, and it was emphasized that the Vitek results were reliable for all strains (including resistant strains). In their study, they found that the overall categorical agreement was 98.3% and the essential agreement was 99% for all test organisms. Buchan et al. 20 , who compared two automated systems and disk diffusion methods to determine the Macrolide-lincosamide-streptogramin B resistance in their study that included 524 staphylococcus strains, reported the susceptibility rates for Phoenix and Vitek 2 systems as 100% and 91.1%, respectively . Gomez-Garces et al 21 . studied one automated system and E test method for four antibiotics including vancomycin in their study that included a total of 180 staphylococci and enterococcus strains, and indicated that there were no very major error or major error between E test and BMD, whereas they found a minor error rate of about 36%. They specified that the automated system they tested as well as E test method exhibited a good agreement with the reference test, except for daptomycin susceptibility of enterococci. In another study, researchers 22 compared the susceptibility results of 150 staphylococci and 51 enterococcus strains obtained from two different automated systems and E test with the results of the reference method, and reported that they found the essential agreement rate of E test in Staphylococci as 96% and in enterococci as 100%. In this study, the number of gram positive bacteria may be higher. In addition, the performance of the tests can also be evaluated in specific resistant strains in our study.
In conclusion, the best agreement rate for all bacteria in our study was found between BMD method and E test. E test is easier to apply and evaluate compared to BMD method. It is a good alternative to BMD method since it also provides the MIC results.
Although our overall agreement rates were over 90% and total error rates were below 10% considering overall agreement rates, the agreement rate of Vitek 2 results was found to be relatively lower than the agreement rates of other two methods. Although we did not evaluate the performance of the tests in this type of strains due to the low rate of specific resistant strains in our study, we are of the opinion that the results of gram negative bacteria for ceftriaxone and nitrofurantoin obtained from automated systems; and the results of ceftriaxone and ciprofloxacin obtained by disk diffusion test should be confirmed by a more reliable method.