Equal Weights vs. Implied Weights in the Lori analysis

While I finish the Ibirania vs. Titanosaurus post, the latest Nanotyrannus paper got me thinking about my resistance to unequal weighting. I learned phylogenetic analysis in the 90s when things were simpler, but over the past several years many papers have used implied weights (IW) for their preferred tress, often relegating equal weights (EW) to the supplementary info. They usually cite Goloboff, so I read his most recent paper on the topic (Goloboff et al., 2018). What the authors did was basically take 158 real morphological datasets ("with 50–149 taxa, and 22–1844 characters; average number of taxa is 80.7, average number of characters is 214.3") and used a Most Parsimonious Tree from each as the 'real' phylogeny, then used those datasets to make 316,000 reduced datasets "with the number of taxa randomly chosen between 40 and 80, number of characters one and half times the number of taxa." These reduced datasets simulate our analyses in the real world that cover only a fraction of real characters and taxa, sort of like how Bootstrap analyses work. Then they checked how closely these reduced datasets matched the 'real' tree using different weighting methods (plus Bayesian and Maximum Likelihood).

What they found was- "The tree inferred by implied weighting is (for almost every combination of taxa, characters and exponential function) more similar to the model tree than the tree inferred using equal weights (Fig. 2e). This difference is overwhelmingly in favour of implied weighting. The number of correct groups retrieved by implied weighting is, for all numbers of taxa, characters, and k, higher than for equal weights (Fig. 2f). The only aspect in which implied weighting performed worse than equal weights is in finding a larger proportion of incorrect groups." The latter sentence seems contradictory, but I take it to mean there were more polytomies in the reduced EW trees, so the IW trees' extra resolved nodes were often wrong. Which if I'm interpreting that right, is a tough blow for those authors who say "our IW results were more resolved, so we're going with those."

In any case, I was curious how the current Lori matrix (700 characters still, but up to 623 taxa with Dilophosaurus as the outgroup) would come out different if I used implied weights k=12 as Goloboff suggests. Both have a similar basic structure in Maniraptoromorpha-

|==coelurid/compsognathid grade
`--+--Ornithomimosauria
   `--+--Therizinosauria
      `--+--Alvarezsauroidea
         `--+--Oviraptorosauria
            `--+--Deinonychosauria
               `--+==basal Avialae
                  `--+--Enantiornithes
                     `--Euornithes

But the differences are interesting (see below for what * means). I'll list some notable ones in three categories-

1. Ways in which EW more closely matches the consensus.
*- Huinculsaurus is an elaphrosaurine, not a noasaurine.
*- Masiakasaurus and Velocisaurus are noasaurines, not basal noasaurids.
*- Koleken is an abelisaurid, not a noasaurid.
*- Piatnitzkysaurids are basal to eustreptospondyline/afrovenatorine-grade megalosauroids, even if they are in the latter clade.
*- Shidaisaurus is a metriacanthosaurid, not a megalosauroid.
*- Xuanhanosaurus and Kaijiangosaurus are metriacanthosaurids, not carcharodontosaurids. Of course these taxa have uncertain placements, but they are always seen as something more basal like metriacanthosaurids, piatnitzkysaurids or non-orionidan tetanurines.
*- Metriacanthosaurids are carnosaurs (and carcharodontosauriforms to boot), not outside Avetheropoda.
*- Tyrannotitan is a giganotosaurine, not a basal carcharodontosaurid.
*- Bicentenaria is a non-tyrannoraptoran coelurosaur, not a proceratosaurid.
*- Chilantaisaurus is a megaraptoran, not a metriacanthosaurid. A controversial taxon, but the former identification has been recovered multiple times, and the latter never has.
*- Murusraptor is a megaraptoran, not a pantyrannosaur.
*- Kileskus is a proceratosaurid, not closer to eutyrannosaurs.
- Harpymimus is outside Garudimimidae+Ornithomimidae, not an ornithomimid.
- Alnashetri is an alvarezsaurid, not an avialan.
- Albinykus is sister to Xixianykus, not closer to derived parvicursorines.
- Protarchaeopteryx is an oviraptorosaur, not sister to Pennaraptora.
- Gigantoraptor is closer to caenagnathoids than to Incisivosaurus.
- Nomingia is a ceanagnathid, not outside Caenagnathoidea. And no, I've scored but not yet integrated the synonymy with Elmisaurus.
- Anomalipes is a caenagnathid, not an oviraptorid.
- Yuanyanglong is sister to Avimimus, not to Apatoraptor.
- Nankangia, Huanansaurus and Corythoraptor are citipatiines, not a basal oviraptorids.
- Fujianvenator is an anchiornithine, not an archaeopterygid.
- Alcmonavis is an archaeopterygid, not a dromaeosaurid.
- Overoraptor is an unenlagiine, not a dromaeosaurine.
- Mahakala is a halskaraptorine, not an alvarezsauroid.
- Yandangornis is outside Jinguofortisidae+Ornithothoraces, not a euornithine.
- Vorona is an ornithothoracine, not a confuciusornithiform.
- Novavis, Shangyang, Yuanjiawaornis, Cuspirostrisornis, Sulcavis, Shenqiornis, Alexornis, Navaornis, Gobipipus, Elektorornis, Longchengornis and Liaoningornis are enantiornithines, not outside Ornithothoraces.
- Protopteryx, Elsornis, Xiangornis, Houshanornis, Noguerornis, Liaoxiornis, Largirostrornis and pengornithids are enantiornithines, not euornithines.
- Avisaurus, Mirarce and Gettyia are grouped closely, unlike IW where the former two are widely separated and the latter is a euornithine.

2. Ways in which IW more closely matches the consensus.
*- Eoabelisaurus and Genyodectes are ceratosaurids, not basal ceratosaurs.
*- Rugops is an abelisaurid, not a ceratosaurid.
*- Xenotarsosaurus and Ekrixinatosaurus are abelisaurids, not noasaurids.
*- Deltadromeus, Kiyacursor and Vespersaurus are ceratosaurs, not ornithomimosaurs.
*- Afromimus is a ceratosaur, not an oviraptorosaur.
*- Poekilopleuron and Erectopus are megalosauroids, not megaraptorans. Like Xuanhano and Kaijiango above, these are both controversial, but always megalosauroid or carnosaurian, not megaraptoran.
*- Wiehenvenator is a torvosaur, not a basal megalosauroid.
*- Shaochilong is a carcharodontosaurid, not a megalosauroid.
*- Tameryraptor is a carcharodontosaurine, not a spinosaurine.
*- Neovenator is a carcharodontosaurid, not an allosaurid.
*- Australovenator and Fukuiraptor are basal coelurosaurs, not carnosaurs (in a topology where megaraptorans are coelurosaurs).
*- Phuwiangovenator is by Fukuiraptor and Australovenator, not a maniraptoran.
*- Santanaraptor is a tyrannosauroid, not a compsognathid.
*- Timimus is a tyrannosauroid, not an ornithomimosaur.
- Tugulusaurus is a maniraptoriform, not a compsognathid.
*- Paraxenisaurus is a deinocheirid, not outside Tyrannoraptora.
- Gallimimus is outside Struthiomimus+Dromiceiomimus, not closer to Struthiomimus.
*- Kol is an alvarezsaurid, not a tyrannosauroid.
- Microvenator is a caenagnathid, not outside Caenagnathoidea.
- Rinchenia is a citipatiine, not a basal oviraptorid.
- Anchiornithids are monophyletic compared to archaeopterygids.
- Jinfengopteryx is a troodontid, not a basal paravian.
- Borogovia is closer to troodontines than sinovenatorines.
- Caihong is an anchiornithid, not a dromaeosaurid.
*- Yurgovuchia is a eudromaeosaur, not a tyrannosauroid.
- Hesperonychus (holotype only) is a microraptorian, not a basal avialan.
- Acheroraptor is close to Velociraptor, not a basal serraraptorian.
- Baminornis is an avialan, not an archaeopterygid (in this tree where Archaeopteryx is not an avialan).
- Linyiornis is an enantiornithine, not outside Ornithothoraces.
- Fortunguavis, Cratoavis and Neuquenornis are enantiornithines, not euornithines.
- Brevidentavis is a hesperornithine, not a troodontid.
*- Gargantuavis is outside Aves, not a palaeognath.
*- Cerebavis is outside Aves, not a neognath.

3. Either alternative seems plausible. Obviously most differences fall into this category, but I found these examples most interesting.
- In EW Gualicho is a 'deltadromean', in IW it's a tyrannosauroid.
- In EW compsognathids are a clade, in IW they're a grade (except Scipionyx and "Sinosauropteryx" lingyuanensis).
- Ornitholestes and Fukuivenator are basal maniraptorans in EW, but just outside Maniraptoriformes in IW.
- In EW scansoriopterygids are basal paravians, in IW they're basal avialans.
- In EW the Ukhaa Tolgod troodontid nestlings (IGM 100/972 and 974) are Almas, in IW they're Byronosaurus.
- In EW Hesperornithoides is a troodontid, in IW it's the basalmost serraraptorian dromaeosaurid.
- In EW Bauxitornis is sister to Balaur, in IW it's an enantiornithine.

Deinonychosauria in EW is-

|--+==Anchiornithinae
|  `--Archaeopteryx
`--+--Unenlagiidae
   `--+--Troodontidae
      `--+--Halszkaraptorinae
         `--Serraraptoria

versus in IW-

|--Halszkaraptorinae
`--+--+--Unenlagiinae
   |  `--Serraraptoria
   `--+--Archaeopteryx
      `--+--Anchiornithinae
         `--Troodontidae

So EW keeps anchiornithines in Archaeopterygidae and halszkaraptorines in Dromaeosauridae, but IW keeps unenlagiines in Dromaeosauridae and anchiornithines in Troodontidae. Basal Avialae in EW is-

|--Zhongornis
`--+--Zhongjianornis
   `--+--Sapeornis
      `--+--Jeholornithidae
         `--+--Confuciusornithiformes
            `--+--Balaur+Fukuipteryx
               `--+--Jinguofortisidae
                  `--Ornithothoraces

versus in IW-

|--Sapeornis
`--+--Zhongornis
   `--+--Zhongjianornis
      `--+--+--Jeholornithidae
         |  `--Balaur+Fukuipteryx
         `--+--Confuciusornithiformes
            `--+--Jinguofortisidae
               `--Ornithothoraces

So there's a consensus that Zhongo, Zhongj and Sape are most basal, then Jehol, Bal+Fuk and Conf are in the next grade, then jinguofortisids sister to Ornithothoraces. The EW tree requires the pygostyle to evolve/reverse one less time. Basal Euornithes (reduced to classic taxa for simplicity) in EW is-

|--+--Patagopteryx
|  `--Archaeorhynchus
`--+--Hongshanornis
   `--+--Yanornis
      `--+--+--Songlingornithidae
         |  `--+--Gansus
         |     `--Ambiortus
         `--+--Apsaravis
            `--Ornithurae

versus in IW-

|--Archaeorhynchus
`--+--Patagopteryx
   `--+--Hongshanornis
      `--+--Yanornis
         `--+--Songlingornithidae
            `--+--Apsaravis
               `--+--+--Gansus
                  |  `--Ambiortus
                  `--Ornithurae

Again, there is consensus of Arch and Patag first, then Hong, then Yan, then some combination of the next four with Gan+Ambi forming a clade, then Ornithurae.

Conclusions- So the * in my lists above indicate taxa that are implicated with non-maniraptoromorph clades or Aves. I think it's notable there's almost twice as many 'correct' placements that involve these in IW compared to EW since these are parts of the tree I didn't add characters for. They just depend on the characters that vary within Maniraptoromorpha that happen to also vary in Ceratosauria, Carnosauria and such. So maybe in areas (or analyses) where characters are poorly sampled, IW does better at matching the 'real' tree. Whereas the EW tree had about twice as many 'correct' placements for the taxa I was actually sampling (since all but two of the enantiornithines that were more basal in IW were in a single clade just outside Ornithothoraces, I only counted that point as three differences). 

Another way to compare the trees is which of the differences in the sampled area do I see as probably wrong, and not just my tree being different but perhaps better than the consensus. For the EW tree, I would say Borogovia, Hesperonychus, Acheroraptor and Brevidentavis are probably in the wrong positions compared to the IW tree. For the IW tree, I'd say Harpymimus, Alnashetri, Protarchaeopteryx, Gigantoraptor, Nomingia, Alcmonavis, Overoraptor, Mahakala, Yandangornis, Novavis and Gobipipus are probably in the wrong positions compared to the EW tree. So the EW tree is almost three times better when it comes to differences I subjectively think are probable errors.

In conclusion, based on this experiment I am surprised at how well the IW k=12 analysis did. I'd even say that if you have an analysis with poorly sampled characters, it may be the way to go. But when it comes to the number of 'better' results in the part of the tree the characters are designed for and the number of differences there that I consider to be mistakes, EW performs about twice and about three times better respectively. So I'm sticking with EW, but I'll keep checking the IW results and will stop criticizing k=12 trees as results that are inaccurate.

Reference- Goloboff, Torres and Arias, 2018 (2017 online). Weighted parsimony outperforms other methods of phylogenetic inference under models appropriate for morphology. Cladistics. 34, 407-437.