Introduction: Urethral stricture is a prevalent condition that often has a major physical and psychological impact on patients. Current treatment includes minimally invasive techniques for mild cases and reconstructive surgery for more severe conditions. However, the objective assessment of surgical outcomes lacks a specific questionnaire validated in Portuguese, although this tool has been available since 2011.

Objective: Perform cross-cultural adaptation and validation of the Urethral Stricture Surgery Patients Reports questionnaire (USS-PROM) into Brazilian Portuguese.

Methods: Three Brazilians fluent in English adapted the questionnaire cross-culturally and the results were compared among them. Three native English speakers fluent in Portuguese translated the resulting version back into English. Face, content, and construct validation were then performed, and agreement and the content validity index (CVI) were calculated. Finally, a pilot validation study was performed using the USS-PROM, the International Prostate Symptom Score (IPSS) and uroflowmetry in 41 patients who underwent urethroplasty at the Hospital do State Public Servant, before and after surgery. Kappa and Spearman coefficients of agreement were used to evaluate the correlation between the USS-PROM, IPSS and urinary uroflowmetry, and Cronbach's alpha coefficient was used to evaluate the reliability and internal consistency of the questionnaire.

Outcomes: Data analysis revealed strong agreement between USS-PROM and IPSS (Kappa of 0.843, p<0.001) and urinary uroflowmetry (Kappa of 0.649, p<0.001). The agreement index was 100% and the CVI was equal to 1. Cronbach's alpha coefficient indicated that the USS-PROM had a high degree of reliability and consistency (0.603 - pre and 0.922 - post).

Conclusion: The USS-PROM questionnaire was validated in Brazilian Portuguese according to the recommended methodology and achieved a high level of agreement and reliability. This allows us to conclude that the USS-PROM questionnaire in Portuguese is a useful, reliable and validated tool for the assessment of patients undergoing urethral stricture correction procedures.

Keywords: questionnaire validation, urethral stenosis and urethroplasty.

Urethral stenosis and strictures consist of a reduction in the caliber of the urethral lumen, causing voiding dysfunction, morphologic and functional disorders of the urinary tract, and compromising the patient's quality of life. It can be a consequence of post-traumatic, post-infectious, idiopathic or iatrogenic causes.^1,2 This condition is probably as old as mankind and is highly morbid, requiring challenging approaches that have evolved significantly in recent decades.

Tissue-transfer urethroplasty with flaps and grafts offers high success rates and is currently considered the technique of choice for the treatment of this condition.¹ However, the evaluation of outcomes and success criteria based on improvement of symptoms and urinary flow are heterogeneous.³

Diagnosis is based on medical history, physical examination and imaging studies such as urinalysis and retrograde cystourethrography. Urodynamic studies, magnetic resonance imaging, ultrasonography and cystoscopy may aid in the diagnosis, but are not essential.⁴

The specific self-report questionnaires (PROMS) are important tools that allow the comparison of preoperative and postoperative scenarios, helping to measure symptoms at these specific moments.^5–7 Nevertheless, these questionnaires need to be validated and adapted to the specific language of the target population, in order to become a reliable tool.⁸

Jackson et al. developed a questionnaire, first validated in England in 2011, to objectively measure outcomes after urethroplasty: the USS-PROM.⁹ The USS-PROM has become an important tool in the assessment of voiding dysfunction in patients with urethral stricture. It is widely used in urological practice and has been validated in several languages, but not yet in Brazilian Portuguese. Therefore, the present study aimed to validate this useful instrument in the Brazilian Portuguese language (Annex 1).

The study was performed on 41 patients based on previous validation articles.¹⁰ The inclusion criteria were adult males with urethral stenosis and spontaneous voiding. This study was approved by the Ethics and Research Committee (number 2.954.555 - October 10, 2018).The statistical analysis was calculated using SPSS 22.0.

After signing the informed consent form, the questionnaire was applied to patients, preoperatively, 3 months and 6 months after urethroplasty, in the Urology Service of the Hospital do State Public Servant of São Paulo, from 2018 to 2020.

The USS-PROM consists of three domains, of which the first six questions have alternatives with scores from 0 to 4, allowing a maximum total of 24 points. It is a reverse questionnaire, since the greater the score, the worst the symptoms. Question 7 is scored from 1 to 4 and assesses how the pathology affects the patient's quality of life. Question 8 is the Peel Chart, which evaluates the strength of the urinary flow according to the distance achieved during micturition. These last two questions are individual scores and do not contribute to the questionnaire's final score.⁹ The second section contains two questions about patient's satisfaction after surgery and the third section includes five questions about the overall health and quality of life.

1. First step: the translation and cross-cultural adaptation

Cross-cultural adaptation and translation was initially performed by three Brazilian English-speaking translators using idiomatic, semantic, and cultural assessments. The resultant translation was retranslated by three native English speakers fluent in Brazilian Portuguese.¹¹

2. Second step: face validity

Face validity was conducted by experts in the field who analyzed the representatively and relevance of the questionnaire domains. After three analyses, the recommended consensus was obtained.^12,13

Subsequently, the Content Validity Index (CVI) and the inter-rater agreement index were calculated. CVI was calculated using a 4-point Likert scale. The evaluators could choose the following answers to rate the question or item's relevance and representatively: 1 = not relevant or not representative, 2 = item needs major revision to be representative, 3 = item needs minor revision to be representative, or 4 = relevant or representative item.^14–16 Comprehensiveness, clarity and relevance were rated on the same scale, and abbreviated options could be presented, such as: 1 = not clear, 2 = somewhat clear, 3 = fairly clear, 4 = very clear.

3. Third step: construct validation and internal consistency

The Cronbach's alpha coefficient was used to evaluate the internal consistency in the preoperative and postoperative periods.¹¹ It is important to notice that the ideal Cronbach's alpha coefficient is greater than 0.70 and considered excellent when greater than or equal to 0.80 (Table 1).

4. Fourth step: Comparison of Validated Questionnaire and Postoperative Uroflowmetric Improvement (Equivalent Forms)

Questionnaire content and voiding symptom grading were compared with the IPSS and uroflowmetry, thereby analyzing the sensitivity of the instruments (capability to identify changes in voiding symptoms after treatment).

Kappa and Spearman's coefficients of agreement were used to test the association between changes observed in the IPSS and uroflowmetry and the ones found in the USS-PROM.

Table 2 presents the demographic profile and Table 3 consists of the original and translated USS-PROM questions, evaluators' observations and statistical analysis. According to Coluci et al,¹⁷ it is recommended that the inter-rater agreement should be above 90%. If this index is not reached, the domain should be reassessed and modified (Table 2&3).

Table 4 illustrates the relevance of each question and the distribution of raters' answers. All items exhibited CVI values of 1.0, exceeding the threshold of 0.7840, which represents an acceptable, as defined by Colucci et al.¹⁷ In the second phase, the evaluators achieved consensus on the introductory text, assigning a score of 4 to all the questions. However, it was recommended that questions 2 and 6 should be revised to enhance their semantics (Table 4).

The analysis of the IPSS in the pre- and post-urethroplasty periods revealed that 82.9% of the patients experienced an improvement in lower urinary tract symptoms, with a mean 15.7-point decrease (p < 0.01).

Uroflowmetry analysis demonstrated an improvement in flow in 87.8% of patients, with a mean increase of 8.9 ml/s in flow rate compared to the preoperative value (p<0.01). The USS-PROM questionnaire indicated a notable improvement in most patients (80.5%), with a mean reduction of 10 points (Tables 5–7& Figure 1).

Figure 1 Percentage of patients who showed improvement, worsening or no change in IPSS, uroflowmetry and USS-PROM after urethroplasty. (Fix overlaps and text in Portuguese).

The observed agreement between the USS-PROM and the IPSS modifications was 95.1%, with a Kappa coefficient value of 0.843. The observed agreement between the USS-PROM and the uroflowmetry improvement was 90.2%, with a Kappa of 0.649 (Table 8).

There was a significant correlation between the USS-PROM and the IPSS, both showing a decrease in scores in the postoperative period, since they are negative questionnaires (p<0.01) in the comparison between the USS-PROM, the IPSS and the urinary flow tools, both in the pre- and postoperative period. A negative correlation was observed between uroflowmetry and USS-PROM scores in the postoperative period. This correlation demonstrated an excellent r-value (Spearman = -0.709). Also there is a correlation between an increase in urinary flow and a decrease in the USS-PROM score. This correlation is illustrated in Tables 7,8, and Figures 2&3 (Table 5-8, Figure 1–3).

Figure 2 Spearman's correlation coefficient = 0.615.

Figure 3 Spearman's correlation coefficient = 0.836.

The Cronbach's alpha statistics applied to the USS-PROM and the IPSS yielded comparable results, indicating that both instruments exhibited a satisfactory degree of reliability and validity for measurement of voiding symptoms (Table 9).

The World Health Organization (WHO) defines health as a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity. However, when assessing subjective data, such as symptoms and well-being, health professionals are challenged to measure improvement or deterioration in these parameters using only the medical history and non-specific questions.

Symptom and well-being assessment questionnaires are therefore essential to quantify the impact of the disease and to adjust patients' expectations regarding the possible results of the proposed treatments. The search for an improvement in quality of life is an important premise of most medical treatments, even if complete cure or the disappearance of pre-existing symptoms are not real possibilities. There are conditions, such as overactive bladder syndrome, where a 50% improvement is acceptable and desired in most treatments.¹⁸

In current urological practice, there are numerous tools for assessing symptoms and quality of life. Specific questionnaires are very useful and bring great benefits to clinical practice and scientific research.¹⁹ Nevertheless, the classic tools for assessing voiding symptoms do not specifically assess symptoms related to urethral strictures.

The USS-PROM is a self-administered questionnaire designed to specifically assess patients' opinion about their voiding status. It is an easy-to-use, concise, practical and well-understood and accepted by patient’s tool¹¹ that thoroughly evaluates the voiding pattern. The eight questions on the questionnaire explore a number of key aspects of the urinary stream pattern, including voiding effort, hesitancy, bladder emptying sensation, quality of life, and the "shape" of the stream (Peeling Graph), truly contributing for medical evaluation.

The reliability of the two tools, USS-PROM and IPSS, was demonstrated by the agreement analysis, which yielded a Cronbach's alpha value of 0.922. In this context, a higher score indicates a more severe degree of voiding dysfunction. The observed agreement between the two questionnaires was 95.1% (Kappa = 0.843), indicating that the questionnaire is an effective tool for detecting voiding symptoms. The Kappa coefficient ranges from 0.4 to 0.6, indicating moderate agreement between the two tools. Values above 0.8 indicate perfect agreement.¹⁴

When the USS-PROM is correlated with uroflowmetry, the relationship is inversely proportional, thereby establishing a standardized concept of flow improvement and score reduction. The postoperative improvement in urinary flow was associated with a reduction in the USS-PROM score, with an observed agreement of 90.2% and a Kappa value of 0.649.

The analysis of the questions among specialists indicated the methodological robustness, reliability and clarity. This was evidenced by the inter-rater agreement index, which was 1, exceeding the 0.78 threshold, and the content validity index, which was above 90%. ^14–16

The analysis of each questionnaire using Cronbach's alpha coefficient revealed a high degree of similarity between the IPSS and the USS-PROM, as both have high and similar indexes. This result indicates the internal consistency, reliability, and applicability of a new tool in comparison to a widely used questionnaire.¹¹

This study covered in detail the scientific methodology recommended for the validation of a questionnaire. It proceeded through a series of steps, including cross-cultural adaptation, face validation, construct validation and internal consistency. Additionally, it considered the tools and methods employed in similar validations conducted in other languages, such as Italian and Spanish.^10,20

The results presented permit the conclusion that the USS-PROM is duly validated in Brazilian Portuguese and can be incorporated in the assessment of voiding symptoms in patients with urethral strictures and stenosis.

The present study employed a robust methodology to validate the USS-PROM questionnaire into Brazilian Portuguese and successfully demonstrated its efficacy, reliability, and ease of understanding. Consequently, this is the inaugural questionnaire developed for the assessment of patients with urethral strictures to be validated into Brazilian Portuguese. The validation of this instrument will contribute significantly to the advancement of Brazilian urological practice.

Annex 1- USS-PROM

Jackson Questionnaire - Thank you for completing this questionnaire. The following questions are designed to measure the effect that urethral strictures have on patients' lives. Some questions may seem the same, but each question is different. Please take time to read and answer each question carefully, and check the box that best describes your symptoms over the past 4 weeks. We are willing to help you respond.

1- Do you take a long time to start urinating?

( ) Never

( ) Rarely

( ) Sometimes

( ) Most of the time

( ) Always

2- Would you say that the strength of your urinary stream is..

( ) Normal

( ) Rarely decreased

( ) Decreased a few times

( ) Decreased most of the time

( ) Always decreased

3- Do you need to make an effort to continue urinating?

( ) Never

( ) Rarely

( ) Sometimes

( ) Most of the time

( ) Always

4- Do you stop and start urinating frequently during your urinary stream?

( ) Never

( ) Rarely

( ) Sometimes

( ) Most of the time

( ) Always

5- How often do you feel like your bladder has not emptied completely after urinating?

( ) Never

( ) Rarely

( ) Sometimes

( ) Most of the time

( ) Always

6- How often have you wet yourself a few minutes after finishing urinating?

( ) Never

( ) Rarely

( ) Sometimes

( ) Most of the time

( ) Always

7- How much does the way you urinate interfere with your life?

( ) Nothing

( ) Little

( ) Moderately

( ) Very

8- Please circle the number that corresponds to the strength of your urinary stream in the last month:

9- Are you satisfied with the result of your operation?

( ) Yes, very satisfied

( ) Yes, satisfied

( ) No, dissatisfied

( ) No, very dissatisfied

10- If you are not satisfied, it is because:

( ) Urinary condition did not improve

( ) The urinary condition improved, but there was some other problem.

( ) The urinary condition did not improve and there was some other problem as well

By marking the questions below, indicate which statements best describe your own state of health today:

11-Mobility

( ) I have no problems walking

( ) I have some problems when walking

( ) I just stay in bed

12- Personal care

( ) I have no problems taking care of myself

( ) I have some problems washing or dressing myself

( ) I can't wash or dress myself

13- Usual activities (e.g. work, study, household chores, family or leisure activities)

( ) I have no problems carrying out my daily activities

( ) I have some problems carrying out my daily activities

( ) I am unable to carry out my daily activities

14- Pain/discomfort

( ) I have no pain or discomfort

( ) I have moderate pain or discomfort

( ) I have severe pain or discomfort

15- Anxiety/Depression

( ) I am not anxious or depressed

( ) I'm a little anxious or depressed

( ) I am extremely anxious or depressed

None.

The authors declare no conflict of interest.

1. Desai D, Martins FE, Tobias SK. Textbook of male genitourethral reconstruction. Springer. 2020.
2. Mathur R, Aggarwal G, Satsangi B, et al. Comprehensive analysis of etiology on the prognosis of urethral strictures. Int Braz J Urol. 2011;37(3):362–369.
3. Anderson KT, Vanni AJ, Erickson BA, et al. Defining success after anterior urethroplasty: an argument for a universal definition and surveillance. Protocol J Urol. 2022;208(1):135–143.
4. Campos-JF, Osman NI, Greenwell T, et al. European association of urology guidelines on urethral stricture disease (part 2): diagnosis, perioperative management, and follow-up in males. Eur Urol. 2021;80(2):201–212.
5. Black N, Jenkinson C. Measuring patients’ experiences and outcomes. BMJ. 2009;339:b2495.
6. N. Devlin. J. Appleby. Getting the most out of PROMs: putting health outcomes at the heart of NHS decision-making. OHE Monograph. 2010.
7. NICE Citizens Council. Quality adjusted life years (qalys) and the severity of illness. National institute for health and care excellence (nice). 2008.
8. Berger M, PN da Luz, Silva NB, et al. Statistical validation of the international prostatic symptom score (I-PSS) in Portuguese. J Bras Urol. 1999;25(2):225–234.
9. Jackson MJ, Sciberras J, Mangera A, et al. Defining a patient-reported outcome measure for urethral stricture surgery. Eur Urol. 2011;60(1):60–68.
10. Barbagli G, Romano G, Sansalone S, et al. [Italian validation of the english PROM-USS-Q questionnaire in patients undergoing anterior urethroplasty]. Urologia. 2011;78(2):98–107.
11. Lucena BSD, Kosminsky M, Costa LJ, et al. Validation of the portuguese version of the RDC/TMD axis II questionnaire. Braz Oral Res. 2006;20(4):312–317.
12. Tilden VP, Nelson CA, May BA. Use of qualitative methods to enhance content validity. Nurs Res. 1990;39(3):172–175.
13. Berk RA. Importance of expert judgment in content-related validity evidence. West J Nurs Res. 1990;12(5):659–671.
14. Wynd CA, Schmidt B, Schaefer MA. Two quantitative approaches for estimating content validity. West J Nurs Res. 2003;25(5):508–518.
15. Rubio DM, Berg-Weger M, Tebb SS, et al. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003;27(2):94–105.
16. S. Summers. Establishing the reliability and validity of a new instrument: pilot testing. J Post Anesth Nurs. 1993;8(2):124–127.
17. Coluci MZO, Alexandre NMC, Milani D. Construção de instrumentos de medida na área da saúde. Ciênc. Saúde Coletiva. 2015;20(3):925–936.
18. Siegel SW, Richardson DA, Miller KL, et al. Pelvic floor electrical stimulation for the treatment of urge and mixed urinary incontinence in women. Urology. 1997;50(6):934–940.
19. C. Kelleher. Quality of life and urinary incontinence. Baillieres Best Pract Res Clin Obstet Gynaecol. 2000;14(2):363–379.
20. Puche-S, Martín-WD, Flores-MJ, et al. Psychometric validation of the spanish version of the uss-prom questionnaire for patients who undergo anterior urethral surgery. Actas Urol Esp. 2016;40(5):322–327.