2 occasions 1 inter-rater ICC

Hello,

I'm working on a reliability study: 10 patients evaluated on two occasions -one week apart - on 8 muscle groups ( 8 parameters by muscle group - 7 quantitative and 1 ordinal ) by 4 raters (2 well experienced to the method and 2 less experienced; even a chronological exeprience from rater 1 to rater 4)
--> 1 intra rater and 2 interrater reliabilities before training --> I use ICCs plus agreememnt frequencies (AF) plus weighted-kappa plus coefficients of variation (CV)

The four raters after the evaluation of these 10 patients are trained to the method and after reevaluated 10 other patients
--> 1 intra rater and 2 interrater reliabilities after training

I would like to compare:
1) impact of training of reliability
2) impact of exprience in using the method

My problems are:
- how to calculate before training interrater ICC taking into account the 2 visits one week apart ? should i only consider the first visit to calculate my interrater ICC ? is it also fair to mean CV, kappa and AF ?
- how can I demonstrate impact of training and/or experience ? should i only consider AF? or each index of reliabilities ? a model is necessary ?


THANK YOU VERY MUCH ++++

Marjo