During this period James had entered protected negotiations with the duke of Longueville to marry his daughter, but these came to nothing. His mistress at this time was Anne Hyde, daughter of Charles's chief minister Edward Hyde, and lady-in-waiting to James's sister Mary. He entered into a private marriage contract with her in Holland in November or December 1659. When news leaked out relatives were horrified. Most refused to acknowledge the marriage, and even James denied it for a while. However, with the Restoration of the monarchy in May 1660, and Anne now heavily pregnant, James went through a public ceremony on 3 September 1660 in London. It was some years, however, before there was any real family harmony.

With Charles installed as king, James had a number of titles bestowed upon him. In addition to Duke of York, he became the duke of Albany and was made an honorary duke of Normandy by the French king, the last English monarch to hold that title. He also became Lord High Admiral. He commanded the navy during the Anglo-Dutch war, defeating the Dutch at the battle of Lowestoft in June 1665. In 1664 the English had captured New Amsterdam in North America from the Dutch. It was presented to James and renamed New York after him.

In 1668 James and Anne converted to Catholicism but kept this secret until Charles was able to force his Declaration of Indulgence through Parliament in March 1672. Anne had died the previous year and James was soon negotiating to marry the ardently Catholic Mary of Modena. The Whig government under the Earl of Shaftesbury was horrified and in March 1673 forced the King to withdraw his Declaration of Indulgence and to pass the Test Act, which banned Catholics from holding public office. James, who had served bravely again in the third Anglo Dutch War at Solebay in May 1672, had to step down as Lord High Admiral. The Government also tried to pass an Exclusion Bill which would have removed James from the succession. Charles was able to thwart this on three occasions between 1679 and 1681, but this and the Popish Plot of 1678 where rumour spread of a plan to assassinate Charles and install James on the throne, caused a massive wave of anti-Catholic feeling. James prudently went into exile in Brussels, and thence to Scotland, in December 1679. He was greeted with considerable caution and trepidation. The Scots had been in upheaval over the last year against potential Catholic reform, and the presence of a Catholic heir in their midst did nothing to pacify them. James, however, remained the soul of discretion for the brief period he was there, and he was generally accepted. However on his return a year later, with the Exclusion Bills rejected, James became less moderate. In July 1681 he forced two bills through the Scottish Parliament. The Act of Succession made it clear that religious differences were no bar to the Succession to the Scottish throne. The Test Act forced all those holding official posts to sign a document pledging their adherence to Presbyterianism, Episcopalianism and Catholicism. Few would comply and most resigned, James using this as an opportunity to purge government of any opposition. Over the next three years, even after he returned South, James continued to persecute the Covenanters and there are many stories (possibly not all true) of the atrocities committed by him and in his name.

These persecutions continued in Scotland after James became king in February 1685. Although he was proclaimed king, he never took the Scottish coronation oath and was the first king of Scotland not to be crowned in Scotland. In fact, he never visited it again. From the moment of his accession there was mounting opposition to James. Neither Scotland nor England wanted a Catholic monarch, especially one with such a cruel and vindictive streak. Two political exiles united with plans to invade their respective countries and oust the king. In Scotland this was Archibald Campbell, earl of Argyll, who invaded Scotland in May 1685, whilst in England it was James Scott, duke of Monmouth, Charles II's eldest illegitimate son, who arrived at Lyme Regis on II June 1685 and was proclaimed king at Taunton nine days later as the real James II. Both men were astonished that so little support rallied to their cause. Argyll was not joined by his clan. His small army was soon overpowered as it marched south. He was captured, imprisoned without trial and executed in Edinburgh on 30 June. Monmouth was defeated at Sedgemoor on 5 July and captured three days later. He was executed at Tower Hill in London on 15 July. The infamous judge Jeffreys was sent on a Bloody Assizes circuit of the West Country, where many of Monmouth's followers were captured and executed.

James now imposed a reign of terror, determined to restore Catholicism in England and Scotland. He introduced a Declaration of Indulgence in April 1687 restoring rights to Catholics, and any protesters were imprisoned, including seven bishops, accused of seditious libel. His complete disregard for the wishes of Parliament and his evident intention to overthrow the Church of England at last galvanised his opponents into action. The last straw was the birth of a son, James on 10 June 1688 after a succession of stillbirths and daughters who had died in infancy. Until then it was possible the church might have tolerated James, as his heir presumptives, Mary and Anne, were both Protestants, but the young James would be raised a Catholic and that prospect was too much. Rumours - false but declared in hope - that the new-born baby was a changeling and that James’s child had died at birth, circulated rapidly and had sufficient credibility to allow the bishop of London and six supporters (known as the "Immortal Seven") to invite James’s son-in-law, William of Orange, to England to protect his wife's succession to the throne. William was himself alarmed at the return of England and Scotland to Catholicism, as it would confirm Britain as an ally of France against the Netherlands and in France's current mobilisation against Germany. On 29 September William sent a declaration to the English Lords accepting their offer and laying out his terms for a "free and lawful Parliament." Delayed at first by bad weather, William's army landed at Brixham on 5 November 1688 and was welcomed at Exeter. Over the next two weeks most of the major cities and bishoprics in England declared their support for William. James's army was first based at Salisbury, but in the light of William's advance he retreated to Reading and called a war council. His commander-in-chief, John Churchill, and others defected to William on the same day, and the next day James discovered that his daughter, Anne, had also defected. Realising he had been deserted, James called a Great Council and agreed to major concessions, including the dismissal of Catholics from office.

Negotiations, led by the Marquis of Halifax, continued for a week, but they were little more than a delaying tactic. On 10 December James fled from London, discarding the Great Seal in the Thames. He was captured at Sheerness in Kent and returned to London, but William allowed him to escape again and he fled to France on 23 December. William accepted the government six days later.

James was installed at the chateau of Saint-Germain, near Paris and established a court-in-exile. With him was his eldest surviving illegitimate son and chief agent and negotiator, James Fitzjames, Duke of Berwick, son of Arabella Churchill, the sister of John Churchill. There were still pockets of support for James, especially amongst the Irish Catholics and the Scottish Jacobites. Following a rebellion amongst the Irish in January 1689, James landed at Kinsale in Ireland and in May 1689 held a Parliament in Dublin which still acknowledged his authority. James raised an army in Ireland, including support from France, and governed for a year, passing a number of acts in favour of the Catholics. In June 1690 William brought his army to Ireland and defeated James at the battle of the Boyne on I July 1690. James fled back to France three days later. He continued to plot and scheme for the next seven years, including a further attempted invasion in 1692 and another planned for but not executed in 1695. He eventually devoted himself to religious pursuits after 1697 and apparently suffered a mental decline. He succeeded in obtaining recognition from France that his legitimate son, James (the Old Pretender), should become king after the death of William III.