| Author |
Message |
< ASReml ~ No covergence even at <1% change in LogL/ var. parameters
|
| mahboob_alam |
 Posted: Thu Jul 26, 2012 1:58 pm |
 |
|
|
Joined: 01 Jun 2009
Posts: 8
Location: Korea
|
Dear Forum members,
I have a case of no convergence ( see attached *.as, *.asr and *.res files). I performed a two-trait anlysis for somatic cell scores. At first very simple models with !GP with US gave me irratic and bouncing LogL values with no convergence attainment. Then I included several qualifiers in JOB file including step size !STEP and others, with an assumption of inappropriate initial values.
Still, I find the repeatation of similar LogL values many times as output which is <1% (even 0%) in terms of %Change in LogL and %Change in variances. In the end, I aborted the JOB using ABORTASR.NOW file forcefully. Please find the details in attached files.
So, my question are:
1) Was that amout change not sufficient to achieve a convergence in ASREML ?
2) Can I use this variance parameters without any question of accuracy?
3) Should I have run few more iterations to get the convergence in the process? or what else I can try to attain convergence?
Any suggestions would be highly appreciated.
Looking forward to you all.
Thankfully,
Mahboob
Post-doc fellow
NIAS, RDA, Korea |
| Description: |
|
Download |
| Filename: |
as_asr_res.rar |
| Filesize: |
4.97 KB |
| Downloaded: |
421 Time(s) |
|
|
| Back to top |
   
|
| Arthur |
| Posted: Sat Jul 28, 2012 2:57 am |
|
|
|
Joined: 05 Aug 2008
Posts: 277
Location: Orange, NSW
|
Dear Marboob,
Sorry but I cant open rar archives, and as there could be many
explanations, I can't comment without looking at the output files.
If you email them to arthur.gilmour@cargovale.com.au, I'll respond to the forum.
Common explanations are
1) a component is small but oscillating slightly so that the convergence criterion is never met but the lack of convergence is inconsequential.
2) some intervention in the AI update which is meant to lead to more
robust convergence is actually putting the updates in some loop; Maybe the model is not appropriate for the data. |
_________________
Arthur Gilmour
Retired Principal Research Scientist (Biometrics) |
|
| Back to top |
|
| Arthur |
| Posted: Mon Jul 30, 2012 2:51 am |
|
|
|
Joined: 05 Aug 2008
Posts: 277
Location: Orange, NSW
|
Dear Mahboob,
In the job you sent, the reason ASReml has not stopped iterating is that the StepSize has not
reached 1.
I have several other comments.
1) I would expect hys age and stage are meant to be factors. Maybe you attempted to specify this with
the asterisk, but it needs to be preceded by a space. i.e.
hys1 * !M0 hys2 * !M0 hys3 * !M0 hys4 * !M0 hys5 * !M0
age1 * !M0 age2 * !M0 age3 * !M0 age4* !M0 age5 * !M0
stage1 * !M0 stage2 * !M0 stage3 * !M0 stage4 * !M0 stage5 * !M0
But maybe the hys factors should be specified as !I 10000
rather than * (assuming there are around 10000 hys classes).
2) I suppose you have added !HOLD to get convergence.
I would not use that. You can see in the .res file that only 4 parameters
are being estimated (numbered 21:24). !hold 15:20 HAS forced the first
two error components to remain at the initial values!
3) Specification of !MVREMOVE with multivariate data can be a problem if it
results in messing up the multivariate pattern of residuals.
==================
I suggest you just use
data_scs_new_analysis.txt !MAXIT 100 !mvinclude !DOPATH $1
after fixing the factor definitions.
When you have that running, if it doesn't converge, let me have another look. |
_________________
Arthur Gilmour
Retired Principal Research Scientist (Biometrics) |
|
| Back to top |
|
| mahboob_alam |
| Posted: Wed Aug 01, 2012 6:07 am |
|
|
|
Joined: 01 Jun 2009
Posts: 8
Location: Korea
|
Dear Professor Aurthur,
Thank you so much for you suggestions. Yes. you were right. I used the !HOLD to get convergence. Anyway, I have tried to change the way you wanted to be. But, my HYS factor seems to be 11 digit integers "11672520001" which looks like as below in data file:
anim sire dam hys1 hys2 hys3 hys4 hys5 ...~ so on
666450 666449 641640 11672520001 11672520011 11672520021 0 0 4 3 2 0 0 4 4 4 0 0 4.570 3.516 1.401
234190 0 0 11673620002 11673620013 0 0 0 7 9 0 0 0 4 2 0 0 0 3.014 3.516 0
So, putting !I 1000000 accounted errors and suggested using !SKIP or !A for hys1 . For this reason I tried using two qualifiers !A 1000000 and !LONGINTEGER 1000000 to avoid 9 digit boundery of !I case and leading it upto 16 digit. But, here I got only convergence for !A case even though same !MAXIT 100 !MVINCLUDE !DOPATH $1 used in datafile line. Well, I haven't seen use of !LONGINTEGER yet. I just found it in the manual. So, not sure, why my data didn't converge in the later case.
Will you guide me here?
And is this a good choice if I use CORGH for residual variances as the data seems to be correlated (repeated measure) with setting necessary !GU, !ASUV, !f mv in the job file?
Attachemts for !A and !LONGINTEGER cases as compiled (i.e. .as, .asr and .res file results) in 1 file as I couldn't attach all 6 of them at a time in one post.
Thank you again, Sir.
Mahboob |
| Description: |
|
Download |
| Filename: |
!A_Log_coverged.dat |
| Filesize: |
12.12 KB |
| Downloaded: |
392 Time(s) |
| Description: |
|
Download |
| Filename: |
Log_not_converged.dat |
| Filesize: |
34.86 KB |
| Downloaded: |
392 Time(s) |
|
|
| Back to top |
|
| Arthur |
| Posted: Thu Aug 02, 2012 10:40 pm |
|
|
|
Joined: 05 Aug 2008
Posts: 277
Location: Orange, NSW
|
Dear Mahboob,
hys1 !A 10000
is the appropriate coding. Where the 10000 number should be adjusted to
the actual number of levels of hys1 (to the nearest 100 or so).
!LONGINTEGER only applies to PEDIGREE factors (it is only defined for the pedigree line).
So, I gather you have the job running now by using hys !A
In the bivariate case, I recommend you use US for the residual variance structure (not CORGH) |
_________________
Arthur Gilmour
Retired Principal Research Scientist (Biometrics) |
|
| Back to top |
|
| Arthur |
| Posted: Thu Aug 02, 2012 10:54 pm |
|
|
|
Joined: 05 Aug 2008
Posts: 277
Location: Orange, NSW
|
I have now looked at your !A output.
Evidently the genetic correlation is very high. Since the variances are very similar, as a reference you could fit the model that assumes the genetic effect is the same. i.e.
... !r Animal .08
1 2 0
0 0 ID
Trait 0 US !GP
3*0
You could also use CORUH for the genetic variance with the original model as that will better allow the correlation to approach 1.
i.e.
Trait.Animal 2
Trait 0 CORUH 0.98 !GP
.085 0.080
Animal 0 AINV |
_________________
Arthur Gilmour
Retired Principal Research Scientist (Biometrics) |
|
| Back to top |
|
| mahboob_alam |
| Posted: Sun Aug 05, 2012 4:06 pm |
|
|
|
Joined: 01 Jun 2009
Posts: 8
Location: Korea
|
Dear Professor Arthur,
I am thankful for your kind suggestions. Both of models worked fine. But my ultimate target was to model 5 SCS traits at a time. Once I changed 2 traits model to 5 traits keeping the same attributes of previous model, my new 5-trait Model seems not progressing after reaching “Forming 7058399 equations: 5 dense” notifications. Even after running it for a long time (more than a day) it cannot go further and cannot complete a single iteration even. I increased memory size (ASREML 3.0, used 16 GB) but no change. Is this happening due to large number of equations or ASREML reaches its maximum limit to conduct such higher level of interactions? I need your help! I find no errors in the model terms.
1) I tried !SM options in the model (splitting the factor fitting upto 6). Singularities is a problem (which converges if I use !AISINGULARITIES and quite faster). But how efficient these estimates are to make a valid conclusion?
Model:
scs1 scs2 scs3 scs4 scs5~ Trait !SM1 at(Tr,1).(hys1 age1 stage1) !SM2 at(Tr,2).(hys2 age2 stage2) !SM3 at(Tr,3).(hys3 age3 stage3), !SM4 at(Tr,4).(hys4 age4 stage4) !SM5 at(Tr,5).(hys5 age5 stage5) !r !SM6 Trait.Animal
(note: i deleted trailing comma in model term here but was in main .as file)
2) What else I can try to make output faster? I was thinking of !MBF qualifier for HYS (hys1, 2, 3, 4, 5) factors. !MBF can work with multilevel factors, although I don’t want to fit HYS as covariates rather as main factors. Can I use !MBF for my data set? I have tried it and faced problem with “no. of levels” as read from mbf file (either high or very low) as well as singularities in this case ;.
Animal !P
sire !M0
dam !M0
age1 * !M0 age2 * !M0 age3 * !M0 age4 * !M0 age5 * !M0
stage1 * !M0 stage2 * !M0 stage3 * !M0 stage4 * !M0 stage5 * !M0
scs1 !M0 scs2 !M0 scs3 !M0 scs4 !M0 scs5 !M0
hys1 !A 1000000 !M0
hys2 !A 1000000 !M0
hys3 !A 1000000 !M0
hys4 !A 1000000 !M0
hys5 !A 1000000 !M0
P_dairy_final.txt !SKIP 1 !MAKE
data_scs_new_analysis2.txt !MAXIT 100 !MVINCLUDE !DOPATH $1
!MBF mbf(hys1,1) MLIB/hys1.csv !sparse
!MBF mbf(hys2,1) MLIB/hys2.csv !sparse
!MBF mbf(hys3,1) MLIB/hys3.csv !sparse
!MBF mbf(hys4,1) MLIB/hys4.csv !sparse
!MBF mbf(hys5,1) MLIB/hys5.csv !sparse
scs1 scs2 scs3 scs4 scs5~ Trait at(Tr,1).hys1 at(Tr,2).hys2 at(Tr,3).hys3 at(Tr,4).hys4 at(Tr,5).hys5 at(Tr,1).age1 at(Tr,2).age2 at(Tr,3).age3 at(Tr,4).age4 at(Tr,5).age5 at(Tr,1).stage1 at(Tr,2).stage2 at(Tr,3).stage3 at(Tr,4).stage4 at(Tr,5).stage5 !r Trait.Animal
-------------------------------------------------------
I set my mbf file like below
hys1.csv (key column,hys_value)
1 1,0
2 1,0
3 1,11690120014
4 1,11690120004
5 1,11690120004
6 1,11690120011
7 1,11690120003
8 1,11690120014
… …
547248 1,0
-----------------------------------------------------
But .asr files showed a different level for HYS factor once I read them from a separate mbf file. Especially for hys2 (1359), hys4 (14), hys5 (16) levels (see attachment plz) which is quite higher if I read the factors from main data file using !A qualifier (You can find it in previous attachemtns):
4 hys1 132036 0 0 1 23218.4811 132035
5 hys2 136202 0 0 1 35691.4410 136202
6 hys3 115507 0 0 2 40225.2373 115507
7 hys4 85777 0 0 1 38700.7156 85777
8 hys5 54029 0 0 1 12956.6285 54029
So, how can the levels can be kept same like output from factors in the main file (just above) if I use !MBF qualifier?
Sorry to bother you a lot. May be my level of english isn't upto the level to understand the manual efficiently.
Hope I will be out of troubles soon.
Thank you again, Professor !! :) |
| Description: |
|
Download |
| Filename: |
MBF_out.asr |
| Filesize: |
12.23 KB |
| Downloaded: |
371 Time(s) |
| Description: |
|
Download |
| Filename: |
After7_hrs_5trait_out.asr |
| Filesize: |
5.84 KB |
| Downloaded: |
370 Time(s) |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum
|
|
|
![[vsni.co.uk]](/common/images/address.gif){kind=small}
Contact us | Site map
News Feed
