I did another quick search for Charles Adam Bylandt, and found another brother of his, Jean Charles Bylandt:
I think I see similarities again.
The nose seems similar, but the eyes and eyebrows are still totally off, so I don't know. (Queen Hortense didn't have those eyes and eyebrows either.)
This is the only Van Bylandt I could find:
http://explore.rkd.nl/nl/explore/portraits/record?filters[RKD_algemene_trefwoorden]=pi%C3%ABdestal&query=&start=25
If there are still male line descendants of these families, it would be a simple enough matter to get them to take dna tests, I suppose. They might even like the connection to royalty, even nouveau royalty.
It seems to be true that it's a wise man who knows his own father...
That said, the world of the 17th, 18th and the very early 19th century aristocrats was very different from the world ushered in by Queen Victoria. I think the men in different countries may have been more or less tolerant, as well. I'm reminded of Alfonso II, Duke of Este, about whom Browning wrote his famous poem "My Last Duchess." In my own area, one of the Malatesta's threw his young wife into a dungeon on suspicion of infidelity. The tower is supposedly haunted by her ghost.

etrified:
The British aristocrat to whom I was referring was Lady Melbourne:
http://en.wikipedia.org/wiki/Elizabeth_Lamb,_Viscountess_Melbourne
"Lady Melbourne had six children that survived childhood; infant twins died in 1788.
[3] Of the remaining six, only the eldest, Peniston, was certain to have been fathered by Lord Melbourne."
The Duchess of Devonshire was also notorious, (there's a wonderful biography of her called Georgiana, Duchess of Devonshire...she comes across as an extremely vulnerable person, more sinned against than sinning...) as was her sister, and Lady Melbourne's daughter in law, Lady Caroline Lamb, wife of the future Prime Minister, who literally lost her mind over Lord Byron.
I think a good deal of this is a not unexpected outcome of marrying young girls to the highest bidder, usually a much older man, rather than on any compatibility whatsoever. And the men, of course, were never expected to be faithful. What's unusual in this class in these particular countries and periods is that the husbands, once in possession of an heir, seem to have turned a blind eye. Of course, if that heir died or was childless, the cuckoo in the nest could indeed wind up inheriting. Probably, up until the late 18th century, it was only the Queens who were watched like hawks.