Case Studies & Client References
- Prevent Critical Defect(s) Escape to Production
- Design Control for OEM and CMO Suppliers
- Supplier Quality Certification
- Offshore Product Development
- Supplier Intended Use Validation – Hawkins
- CyberSecurity Test Support – Packetsled
- FDA Submittal – Oxford Immunotec
- SDLC Process Improvement – Baxter Healthcare
- QMS Gap Analysis – Synthetic Genomics, Inc
- BioTelemetry Technology
Medical Device Risk, Patient Safety and Hospital System Performance
Employed active Risk Reduction to identify and mitigate highest priority “what if” scenarios
Built comprehensive test plan to catch critical and high priority defects early & often
Eliminated bottlenecks to increase patient safety
Project: New centralized system for monitoring medical devices throughout a hospital, (connecting large quantities of embedded medical delivery systems with server based technologies), for the first time.
Concern: Marketing was concerned with critical data flows and connectivity, while Engineering was focused on system performance and potentially degraded user experience. Because inherited code was used to build the browser-based user interface, and these medical devices had never been networked across a clinical environment – the main concerns were system integration and performance, both for server software and the devices themselves. The project was on a strict timeline, and because of the regulated environment we operated in, there was zero tolerance for patient risk due to data loss or system degradation.
Problem: Early Performance Testing exposed multiple weaknesses, such as unacceptable user interface response times, intermittent connectivity between medication delivery devices and the server system, and painfully slow device and performance measurement data flow.
Solution: Due to time constraints, the Development and Software Quality Teams determined the direct path to success would be the utilization of Risk Management processes.
By identifying the high risk areas of the code early – via multiple Impact vs Probability sessions where we created hundreds of “what if” scenarios, then narrowed them down to the highest impact areas – we were able to create test cases designed to expose high severity defects early and often.
Additionally, after reviewing these high risk areas of the system – we identified expected bottlenecks, which allowed us to focus our performance test activities on appropriate interfaces, server capabilities and loads generated by high quantities of connected devices.
Working with the DB Developers, we utilized additional tools readily available, which allowed us to monitor and record server resources, and troubleshoot performance issues to their root cause, in near real time. This increased fix build velocity and enabled earlier project release.
Successful Results: By identifying major bottlenecks within core features, we were able to prevent potentially catastrophic consequences in the hospital environment.
Extended Benefits: By taking advantage of open source tools such as JMeter, we were able to certify them for use in our regulated environment, training time was minimal, and we were able to build a robust, re-usable performance testing program which could be utilized across numerous projects.
Design Control for OEM and CMO Suppliers
Created engineering change procedure that accommodated all engineering change types
Created on workflow to accommodate all change types
Discovered (then closed) Gaps in order to improve manufacturing process and product quality
Project: Perform analysis of Invitro-diagnostic device manufacturer’s Design Control and Engineering Change Management (ECR, ECO, ECN) processes for OEM and Contract Manufacturing. DHF design transfer documentation was not current, and DMR’s did not reflect all ECO’s..
- Requested current SQA’s and SLA’s with OEM and Contract Manufacturers
- Requested training records of OEM and CMO personnel to client SOP’s
- Reviewed pFMEA of ECO, Release to manufacturing, and Doc Control process for Engineering Drawings and Software
- Review ECO’s cut in by OEM and CMO’s
- Reviewed existing Design Control and Supplier Procedures
Results of Review:
- No approved Supplier Quality Agreement’s or Service Level Agreement’s
- SQA’s hung up in legal department
- No training records for OEM and CMO personal
- Training plans for OEM and CMO were not available or missing
- Incomplete pFMEA for device manufacturing
- pFMEA was initiated by manufacturer and was to be completed by CMO
- Changes to QMS procedures for Design Control, Risk management, and Supplier Quality were in a state of ‘flux’
- Quality group has made changes to Design Control and Risk Management procedures and Supplier Quality procedures
- Documents are being submitted to two different document control systems
- One system for engineering documents and the other for QMS & Quality record documentation
Benefits: Recommendations Successfully Implemented
- Review quality agreements with OEM and CMO’s. Manufacturer’s processes to be identified and implemented at CMO facility. ECO to be approved by Manufacturer with ECN’s sent to CMO for implementation, cut in dates and lot ID. Manufacturer maintains ECO and design history records while CMO updates DMR’s and provides updates to Manufacturer Quality for review
- OEM and CMO’s to train to Manufacturer’s procedures and work instructions. Training records to be maintained by Manufacturer
- Review design control, risk management, and supplier quality procedures. Created one Engineering Change procedure to accommodate all engineering change types and one workflow to accommodate all change types. Make updates as necessary for compliance to ISO-13485:2016, 21CFR830, ISO-14971:2012. Ensure quality agreements are in alignment
- Reviewed ECO’s with design impact for potential hazards. Reviewed System Hazard Analysis, Device FMEA, and updated based on findings from past ECO’s. Updated Risk file. Reviewed pFMEA, updated. CMO performs pFMEA with guidance from Manufacturer
- Reviewed doc control procedures and updated. Identified one doc system as Device Engineering and Manufacturing Doc Control and submitted all engineering documentation. Second system used to maintain Quality System
Design Controls for CMO Suppliers
Project: Project involved upgrading and formalizing Supplier Quality Certifications and Training. Receiving Inspections were discovering an excessive number of incoming parts being rejected
Process: Advantu Process Expert Strategy for Gap Analysis Statement of Work
We have found the most productive SOW engagements are those that are defined through an in-depth Gap Assessment and Risk Based Prioritization. This is needed to gain a comprehensive understanding of not only the current state but of the desired state. Through this interaction we are better able to address the most cost-effective changes for you the client. The basic assessment flow is as follows:
A. Initial Assessments to Identify Possible Gaps
B. Risk Assessment and Cost Benefit Analysis of Identified Gaps
C. Prioritize Gap Assessment Findings
D. Define Statement of Work based upon Prioritized Gap Categorization
A. Initial Assessments (Recommended but determined through Collaboration) to Identify Possible Gaps:
1. Supplier Certification Procedure
2. Supplier Certifications (Current and Planned)
3. Supplier Process Validation Procedure and Records
4. Supplier Monitoring Process and Records
5. Management Reviews
6. Internal Audits
8. Cost of Quality Metrics
9. External Audits
11. Supplier Manufacturing Procedures
12. Manufacturing and Sourcing Procedures and Records
13. Manufacturing Process Inspection Procedures and Records
14. Feature Inspection Plan Procedures and Records
15. Incoming Inspection Procedure and Records
16. Gage R&R Procedure
17. Critical to Quality Identification
18. Change Management Procedure
19. Metrology Procedure and Records
20. Health Hazard Analysis Procedure and Records
21. Supplier Quality Training
22. Risk Management Procedure and Training Records
23. Production Transfer Procedures and Records
a. From Production Sites
b. To Production Sites
24. DHF Procedure and Records
25. Lot Control Procedure and Records as Applicable
B. Risk Assessment and Cost Benefit Analysis of Identified Gaps
2. Risk Type
4. Likelihood of Occurrence
5. Possible Mitigations
C. Prioritize Gap Assessment Findings Based Upon Risk
1. Categorize Prioritized Gaps
• Near Term
• Long Term
D. Define Statement of Work based upon Prioritized Gap Categorization
1. Process Flow (PERT)
2. Identify Successor Tasks
3. Identify Predecessor Tasks
4. Identify Task Division of Responsibility
5. Identify Required Inputs
6. Identify Planned Outputs
7. Identify Needed Approvals
Problem: Shortly after the go-live date, the system began to experience degrading performance under normal load. The software publisher suggested upgrading system hardware in order to stem performance degradation. This solution proved short lived, as the system once again began to experience slow-down under normal load.
Due to the required path forward, an approach to salvage the new system was needed.
Solution: The Development and Software Quality Teams developed a system functionality matrix to isolate possible areas of performance impact. Development and Software Quality Engineers teamed up in pairs to analyze functional areas under load. These teams interviewed users to help isolate problem areas in the system.
By assigning specific areas of the system to functional experts, it was possible to perform both static analysis and verify transactional outcomes quickly.
A number of tools were employed to probe the code execution and database transactions. Some were developed specifically for this project by the Development Team.
The testing procedure yielded a number of improvements that could be made including a database call that was being made from more than one code block. This call included a table join that resulted in a Cartesian product, the main source of the physical resource overload.
Successful Results: The team worked with the publisher to provide the details required to submit bug fix requests, and the resulting software patch restored the system to a desired level of usable performance.
Extended Benefits: The pairing of Development and Software Quality Engineers with functional expertise became a common approach to troubleshooting systems that were experiencing issues of performance and unexpected results. Shared domain knowledge has helped provide a path to resolving issues by allowing users and engineers to communicate more freely.
Offshore Implementation and Execution
Capacity & Efficiency Strategy
Designed and Documented Outsource Plan (internal/external teams)
Identified critical gaps in communication, domain knowledge transfer, expectations for successful deliverables, and status reporting
Executed Outsource Plan for multiple projects simultaneously, utilizing ongoing accountability with Offshore Team for a successful release
Project: New hospital system for use in Europe, with heavy utilization of offshore test team working with internal resources, on a set of platforms being integrated for the first time.
Concern: Completely new team, offshore and internal – require extensive training for product knowledge and FDA Regulations and company procedures – on a tight project schedule.
Additionally, language barriers with offshore teams were a strong concern across all teams – especially engineering and marketing, where daily communications are expected during the entire project.
Finally – the time difference between internal and offshore teams created an initial communication dilemma.
Problem: Early on, when meeting with established teams – via teleconference – language barriers prevented successful communication of tasks and deliverables.
The Engineering team was also concerned with the caliber of the external testing team and highlighted the need for technical team interviews, including assessing their knowledge and culture fit to maximize project success.
Additionally, establishing the communication style to guarantee understanding of assignments, and confirmation of successful deliverables, without total micromanagement.
Finally, lack of background in the medical device realm and lack of understanding of regulated processes increased the training time dramatically.
Note: when ordering equipment for testing purposes – customs delays on both sides – directly impacted the project schedule, so ordering well in advance of project launch is a critical task.
To overcome the initial language barrier and build trust, we established the following protocol:
Identify Leads with strong English communication skills
Determine the most effective communication tool (phone, skype, IM chat, email)
Social interactions maximized team building
To establish a process, to ensure project remains on track, and address issues in a timely manner:
Communicate project need-fuls and tasks to the Leads/Team
- Via email – team shall confirm all tasks and expected deliverables
- Daily standup, same time every day – internal lead confirms previous day’s deliverables, review blocking issues, review tomorrow’s tasks with all team members
- At the end of each work day – following internal team template – offsite team was required to provide complete details of daily assignments and deliverables
To verify the caliber of external test team, the internal lead identified the necessary project skill-sets, and conducted team interviews for best fit.
To fill the knowledge gap, for medical device and regulated processes – we conducted extensive product, tools & process training, including ongoing e-classes which conformed to FDA regulations and company policies.
All offshore Leads were fully trained onsite and were provided all literature required to transfer knowledge to their teams back home.
Because the Senior onsite QA Lead took on the role of daily communication with the offshore PM, and oversaw all QMS Processes internally – the test team was able to focus on test planning, risk identification -> what if scenarios -> test case creation, and test execution, allowing the project leads to identify all high severity defects early and often.
The project was released successfully, meeting all project goals and most importantly – all customer expectations were met at launch, with no major issues reported and contributed to the green light for the follow on project.
Additionally, this approach allowed the combined teams to successfully execute on multiple projects simultaneously, moving forward.
After establishing the successful process and expectations of the Offshore team, we were able to use these processes for our future projects in less time and with greater impact.
*Note: due to confidentiality agreements in place, all clients listed in our Case Studies shall remain confidential.
“We wanted to update and improve our Intended Use Validations at Hawkins for both COTS and Custom developed applications. Included in this effort was the Electronic Signature (21CFR Part 11) validation of our internally developed system supporting our QMS.’
‘We brought Advantu’s Process Experts on board. They quickly conducted assessments of several Hawkins’ systems and efficiently identified Intended Use Validation gaps that needed to be resolved in support of our validation efforts. Additionally, we were provided a plan of action to close the identified gaps.’
‘Advantu’s team was extremely professional, knowledgeable and enjoyable to work with. Their deliverables were completed professionally and provided the foundation for the next steps in our IUV process. We look forward to our continued relationship with Advantu’s experts.’”
PacketSled wanted to increase product scalability for their larger clients; Advantu’s Performance Consultant reviewed their back-end systems, and created an improvement plan which PacketSled’s engineering team is utilizing moving forward.
Additionally, per request, Advantu sourced a top notch DevOps Engineer who has performed exceptionally.
Per Troy: “I’ve had nothing but great experiences working with Kris, and he is absolutely correct in stating that he was able to efficiently get us the solution we needed almost immediately.”
Director, Program Management
Advantu provided a team for software quality assurance and testing support, including quality engineering (regulatory) support for an FDA submittal.
Advantu’s team enabled OI to meet the stringent FDA demands for a successful submittal, and the critical deadlines for two (client) clinical trials.
Vice President, Device Engineering, Hospital Products
Every day we are challenged with finding ways to improve quality of software development while decreasing time-to-market. Specific areas we focused on were automated tools for development and test, increased transparency and traceability, automated regression analysis and repeatable predictable outcomes.
Charlie Gragg and his team assessed our current development and test situation and presented the best path forward for addressing our specific needs. If you have a similar need, Charlie is the right resource to meet your requirements. I highly recommend him and his team.
Manager, Quality Systems
Synthetic Genomics, Inc.
We wanted to add a new DNA Synthesis Instrument to our GMP workflow, so we brought Advantu’s Process Experts on board to conduct the appropriate analysis, develop the plan and spearhead the implementation.
Advantu’s team was extremely professional, knowledgeable, enjoyable to work with and ensured we had everything we needed. Advantu’s deliverables were completed ahead of schedule, resulting in a very successful project.
Vice President, Research and Development
I hired Patrick Ward at Hospira as my replacement to lead the 2 Java teams. From the day that he stepped foot in the door, Patrick exhibited a dedication and drive for success that I had never experienced before.
He poured his blood, sweat and tears into everything he did for the success of his many concurrent projects and for the care of his team members. He consistently applied his keen business acumen to look for opportunities to target new market segments, decrease support costs, maximize product flexibility and create better value for his customers.
I am better for having worked with him. Patrick will be an indispensable asset to every organization he joins.