Case Studies & Client References
- Medical Device Company
- Cyber-Security Firm
- Agro-Industrial Company
- Multinational Hospital Systems Company
Medical Device Risk, Patient Safety and Hospital System Performance
Employed active Risk Reduction to identify and mitigate highest priority “what if” scenarios
Built comprehensive test plan to catch critical and high priority defects early & often
Eliminated bottlenecks to increase patient safety
Project: New centralized system for monitoring medical devices throughout a hospital, (connecting large quantities of embedded medical delivery systems with server based technologies), for the first time.
Concern: Marketing was concerned with critical data flows and connectivity, while Engineering was focused on system performance and potentially degraded user experience. Because inherited code was used to build the browser-based user interface, and these medical devices had never been networked across a clinical environment – the main concerns were system integration and performance, both for server software and the devices themselves. The project was on a strict timeline, and because of the regulated environment we operated in, there was zero tolerance for patient risk due to data loss or system degradation.
Problem: Early Performance Testing exposed multiple weaknesses, such as unacceptable user interface response times, intermittent connectivity between medication delivery devices and the server system, and painfully slow device and performance measurement data flow.
Solution: Due to time constraints, the Development and Software Quality Teams determined the direct path to success would be the utilization of Risk Management processes.
By identifying the high risk areas of the code early – via multiple Impact vs Probability sessions where we created hundreds of “what if” scenarios, then narrowed them down to the highest impact areas – we were able to create test cases designed to expose high severity defects early and often.
Additionally, after reviewing these high risk areas of the system – we identified expected bottlenecks, which allowed us to focus our performance test activities on appropriate interfaces, server capabilities and loads generated by high quantities of connected devices.
Working with the DB Developers, we utilized additional tools readily available, which allowed us to monitor and record server resources, and troubleshoot performance issues to their root cause, in near real time. This increased fix build velocity and enabled earlier project release.
Successful Results: By identifying major bottlenecks within core features, we were able to prevent potentially catastrophic consequences in the hospital environment.
Extended Benefits: By taking advantage of open source tools such as JMeter, we were able to certify them for use in our regulated environment, training time was minimal, and we were able to build a robust, re-usable performance testing program which could be utilized across numerous projects.
Software Quality Process Improvement
Reviewed client’s software development process
Introduced configuration management, smoke test, code delivery rules to build release process
Advantu built a comprehensive test plan to catch critical defects early and often
Project: Cloud based network scanning application designed to analyze, identify and report myriad threats, malware intrusions and denial of service attacks across the client’s systems in real time.
Concern: The business wanted to increase the coverage of their quality assurance and testing program and requested a professional review of their entire software delivery process, in order to reduce risk and increase confidence with upcoming releases.
Problem: Initial deployments with smaller clients revealed issues which should have been discovered prior to release, and had the potential to increase risk with future releases.
Solution: Advantu’s QA Consultant reviewed their entire process to identify all areas of risk, including recommendations for remediation and ongoing process improvement.
Successful Results: Our QA Consultant worked with their team to successfully implement an improved QA Program…which increased coverage, throughput and markedly improved customer satisfaction with the application.
Data Reliability and System Performance Validation
Paired Development and Software Quality Engineers with functional experience
Conducted team interviews of end users to isolate problem areas quickly
Assigned specific system areas to functional experts enabled rapid static analysis and transactional outcome verification
Project: A new ERP system for tracking sales, planning and financial details of a multi-grower agricultural association had been implemented by the publisher of the system.
Concern: Reliability and System Performance.
Management was concerned with integrating existing data with the new system to provide historical information as well as provide a basis for budgeting and planning. Management was also concerned about the reliability of the new system; given that this was the first large scale implementation of this industry-specific ERP.
The project was on a strict timeline imposed by the imminent failure of the existing system. There existed no fail-over system to capture critical business data in the case of system degradation or failure.
After initial setup was deemed successful, historical data was loaded and testing began.
Problem: Shortly after the go-live date, the system began to experience degrading performance under normal load. The software publisher suggested upgrading system hardware in order to stem performance degradation. This solution proved short lived, as the system once again began to experience slow-down under normal load.
Due to the required path forward, an approach to salvage the new system was needed.
Solution: The Development and Software Quality Teams developed a system functionality matrix to isolate possible areas of performance impact. Development and Software Quality Engineers teamed up in pairs to analyze functional areas under load. These teams interviewed users to help isolate problem areas in the system.
By assigning specific areas of the system to functional experts, it was possible to perform both static analysis and verify transactional outcomes quickly.
A number of tools were employed to probe the code execution and database transactions. Some were developed specifically for this project by the Development Team.
The testing procedure yielded a number of improvements that could be made including a database call that was being made from more than one code block. This call included a table join that resulted in a Cartesian product, the main source of the physical resource overload.
Successful Results: The team worked with the publisher to provide the details required to submit bug fix requests, and the resulting software patch restored the system to a desired level of usable performance.
Extended Benefits: The pairing of Development and Software Quality Engineers with functional expertise became a common approach to troubleshooting systems that were experiencing issues of performance and unexpected results. Shared domain knowledge has helped provide a path to resolving issues by allowing users and engineers to communicate more freely.
Offshore Implementation and Execution
Capacity & Efficiency Strategy
Designed and Documented Outsource Plan (internal/external teams)
Identified critical gaps in communication, domain knowledge transfer, expectations for successful deliverables, and status reporting
Executed Outsource Plan for multiple projects simultaneously, utilizing ongoing accountability with Offshore Team for a successful release
Project: New hospital system for use in Europe, with heavy utilization of offshore test team working with internal resources, on a set of platforms being integrated for the first time.
Concern: Completely new team, offshore and internal – require extensive training for product knowledge and FDA Regulations and company procedures – on a tight project schedule.
Additionally, language barriers with offshore teams were a strong concern across all teams – especially engineering and marketing, where daily communications are expected during the entire project.
Finally – the time difference between internal and offshore teams created an initial communication dilemma.
Problem: Early on, when meeting with established teams – via teleconference – language barriers prevented successful communication of tasks and deliverables.
The Engineering team was also concerned with the caliber of the external testing team and highlighted the need for technical team interviews, including assessing their knowledge and culture fit to maximize project success.
Additionally, establishing the communication style to guarantee understanding of assignments, and confirmation of successful deliverables, without total micromanagement.
Finally, lack of background in the medical device realm and lack of understanding of regulated processes increased the training time dramatically.
Note: when ordering equipment for testing purposes – customs delays on both sides – directly impacted the project schedule, so ordering well in advance of project launch is a critical task.
To overcome the initial language barrier and build trust, we established the following protocol:
Identify Leads with strong English communication skills
Determine the most effective communication tool (phone, skype, IM chat, email)
Social interactions maximized team building
To establish a process, to ensure project remains on track, and address issues in a timely manner:
Communicate project need-fuls and tasks to the Leads/Team
- Via email – team shall confirm all tasks and expected deliverables
- Daily standup, same time every day – internal lead confirms previous day’s deliverables, review blocking issues, review tomorrow’s tasks with all team members
- At the end of each work day – following internal team template – offsite team was required to provide complete details of daily assignments and deliverables
To verify the caliber of external test team, the internal lead identified the necessary project skill-sets, and conducted team interviews for best fit.
To fill the knowledge gap, for medical device and regulated processes – we conducted extensive product, tools & process training, including ongoing e-classes which conformed to FDA regulations and company policies.
All offshore Leads were fully trained onsite and were provided all literature required to transfer knowledge to their teams back home.
Because the Senior onsite QA Lead took on the role of daily communication with the offshore PM, and oversaw all QMS Processes internally – the test team was able to focus on test planning, risk identification -> what if scenarios -> test case creation, and test execution, allowing the project leads to identify all high severity defects early and often.
The project was released successfully, meeting all project goals and most importantly – all customer expectations were met at launch, with no major issues reported and contributed to the green light for the follow on project.
Additionally, this approach allowed the combined teams to successfully execute on multiple projects simultaneously, moving forward.
After establishing the successful process and expectations of the Offshore team, we were able to use these processes for our future projects in less time and with greater impact.
*Note: due to confidentiality agreements in place, all clients listed in our Case Studies shall remain confidential.
PacketSled wanted to increase product scalability for their larger clients; Advantu’s Performance Consultant reviewed their back-end systems, and created an improvement plan which PacketSled’s engineering team is utilizing moving forward.
Additionally, per request, Advantu sourced a top notch DevOps Engineer who has performed exceptionally.
Per Troy: “I’ve had nothing but great experiences working with Kris, and he is absolutely correct in stating that he was able to efficiently get us the solution we needed almost immediately.”
Manager, R&D Program Management
Shawn was managing multiple programs, while working with our Software Quality Engineer Eleonora (Lina) Kaziyeva, who was leading the Quality Assurance efforts, which ultimately led to successful conclusions for every program she was asked to cover.
Per Shawn: “Eleonora had numerous assignments that went above and beyond her role including coordination of team members from multiple companies/vendors in a design center spread across the globe, working in multiple quality management systems as well as guiding the team for compliance, to both domestic and international regulations, and ensuring that the timing, accuracy and completeness of the software product met and exceeded expectations.”
Additionally: “Based on her work I would rank her as one of the best software quality engineers I have ever had the pleasure of working with.”
Software Test Engineer IV
RE: Advantu QA Training Course
“Overall the class is very informative, good content and handout materials are very good…
The Instructor was very knowledgeable about the topic. I would definitely recommend the class to others !”
Technical Project Manager
Oracle EBS Project
The Contract QA Engineer assigned to this project, Kris Kelly, was tasked with establishing and leading a QA Team in order to implement industry standard software testing activities – and increase the potential for a preexisting project to meet client needs for usability, reliability and data accuracy.
Kris immediately reached out to the program management team – including the software development team – in order to fully evaluate the project requirements, tools and skill sets required, communications protocols and expected deliverables including scheduling.
Based upon his findings, Kris interviewed and selected the offshore testing team, and immediately set up a brief training program, designed to rapidly increase domain knowledge – and allow the test team to ‘hit the ground running’ in order to support the project timeline.
During his contract tenure, Kris was able to evaluate the project needs, including personnel skill sets and temperament and build a team which was able to successfully deliver on time and with the highest product quality.
Vice President, Research and Development
I hired Patrick Ward at Hospira as my replacement to lead the 2 Java teams. From the day that he stepped foot in the door, Patrick exhibited a dedication and drive for success that I had never experienced before.
He poured his blood, sweat and tears into everything he did for the success of his many concurrent projects and for the care of his team members. He consistently applied his keen business acumen to look for opportunities to target new market segments, decrease support costs, maximize product flexibility and create better value for his customers.
I am better for having worked with him. Patrick will be an indispensable asset to every organization he joins.